Third Light Developer Exchange

paulblighe

Member

Registered: 2014-11-11

Posts: 8

.Net Based File Upload

I am trying upload a file to a folder uisng the json api. I have successfully created a folder using the api but cannot then add content to the folder.

I am logging in using the Core.LoginWithKey function and the api key that is set up in our account.

What are the steps I need to take to achieve this. I am creating an upload using Upload.CreateUpload and successfully getting back the upload key and post url. I am then adding files to the upload using Upload.AddFilesToUpload (I am assuming I am using the post url in the web request), at this point I am unsure if it has worked as within the response stream I am getting a html page returned. I have then called Upload.StartUpload (again do I use the post url returned from the CreateUpload as the web request url?). Again it doesn't look like this has worked.

Using the api what calls do I need to make to get a successful upload

The application I am using is a .net console application. Any help or pointers in how to upload a document to a newly created folder would be appreciated.

paulblighe

Member

Registered: 2014-11-11

Posts: 8

Re: .Net Based File Upload

Ok so I am getting nowhere fast with this.  Here are the steps I have taken based on the php example found in the integration documentation.

Step 1 - Login, which returns a sessionId successfully

var httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
                httpWebRequest.ContentType = "application/json";
                httpWebRequest.Method = "POST";

                using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
                {
                    string json = "{\"action\":\"Core.Login\"," +
                                  "\"apiVersion\":\"1.0\"," +
                                  "\"inParams\":{\"username\":\"" + username + "\", \"password\":\"" + password + "\"}}";

                    streamWriter.Write(json);
                }
                var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
                using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
                {
                    result = streamReader.ReadToEnd();            
                }

Step 2 - Add a folder, which returns the foldeId (amongst other information) successfully

var httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
                httpWebRequest.ContentType = "application/json";
                httpWebRequest.Method = "POST";

                using (var streamWriter1 = new StreamWriter(httpWebRequest.GetRequestStream()))
                {
                    string json = "{\"action\":\"Folders.CreateFolder\"," +
                                   "\"apiVersion\":\"1.0\"," +
                                   "\"sessionId\":\"" + sessionId + "\"," +
                                   "\"inParams\":{\"folderName\":\"" + folderName + "\"," +
                                   "\"parentId\":\"" + parentFolder + "\"," +
                                   "\"type\":\"FOLDER\"," +
                                   "\"options\":\"\"," +
                                   "\"searchParams\":\"\"" +
                                   "}" +
                                    "}";

                    streamWriter1.Write(json);
                }
                var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
                using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
                {
                    result = streamReader.ReadToEnd();            
                }

Step 3 - Create the upload, which returns the uploadkey and PostUrl successfully

var httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
            httpWebRequest.ContentType = "application/json";
            httpWebRequest.Method = "POST";

            using (var streamWriter1 = new StreamWriter(httpWebRequest.GetRequestStream()))
            {
                string json = "{\"action\":\"Upload.CreateUpload\"," +
                               "\"apiVersion\":\"1.0\"," +
                               "\"sessionId\":\"" + sessionId + "\"," +
                               "\"inParams\":{\"params\":{\"destination\":\"" + parentId + "\"," +                              
                               "\"synchronous\":true," +
                               "\"blocking\":true," +                               
                               "\"lifetime\":3600}" +
                               "}" +
                                "}";

                streamWriter1.Write(json);
            }
            var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
            using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
            {
                returnValue = streamReader.ReadToEnd();         
            }

Step 4 - Add files to upload, it is at this point I don't get the expected return

FileStream fs = new FileStream(fileName,
                                    FileMode.Open,
                                    FileAccess.Read);
                byte[] filebytes = new byte[fs.Length];
                fs.Read(filebytes, 0, Convert.ToInt32(fs.Length));
                string encodedData =
                    Convert.ToBase64String(filebytes,
                                           Base64FormattingOptions.None);

                var httpWebRequest = (HttpWebRequest)WebRequest.Create(urlToUse);
                httpWebRequest.ContentType = "application/json";
                httpWebRequest.Method = "POST";

                using (var streamWriter1 = new StreamWriter(httpWebRequest.GetRequestStream()))
                {
                    string json = "{\"apiVersion\":\"1.0\"," +
                                   "\"sessionId\":\"" + sessionId + "\"," +
                                    "\"action\":\"Upload.AddFilesToUpload\"," +
                                   "\"inParams\":{\"uploadKey\":\"" + uploadKey + "\"," +
                                   "\"fileData\":{" +
                                   "\"fileOne\":{" +
                                   "\"encoding\":\"base64\"," +
                                   "\"name\":\"cow_in_field.jpg\"," +
                                   "\"data\":\"" + encodedData + "\"" +                                  
                                   "}" +
                                   "}" +
                                    "}}";

                    streamWriter1.Write(json);
                }
                var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
                using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
                {
                    result = streamReader.ReadToEnd();
                    //Now you have your response.
                    //or false depending on information in the response                
                }

As I am using a synchronous call and blocking is set to true I am expecting a succeeded key to be returned but instead I get the following html in the request stream.

<!DOCTYPE html>
<!--[if IE 8]>                  <html class="ie8" lang="en"> <![endif]-->
<!--[if IE 9]>                  <html class="ie9" lang="en"> <![endif]-->
<!--[if (gt IE 9)|!(IE)]><!-->  <html class="" lang="en"><!--<![endif]-->
<head>
    <meta name="Generator" content="Third Light Ltd Intelligent Media Server" />
    <meta charset="UTF-8" />
    <meta http-equiv="Content-Type" content='text/html; charset=UTF-8' />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
            <!--[if !IE]> --><link rel="stylesheet" href="css_parser.tlx?v=06010205-1408612672&amp;themeid=1&amp;mode=full" type="text/css" /><!--<![endif]-->
        <!--[if gt IE 8]><link rel="stylesheet" href="css_parser.tlx?v=06010205-1408612672&amp;themeid=1&amp;mode=full" type="text/css" /><![endif]-->
        <!--[if lte IE 8]><link rel="stylesheet" href="css_parser.tlx?v=06010205-1408612672&amp;themeid=1&amp;mode=ie" type="text/css" /><![endif]-->
            <link rel="shortcut icon" type="image/x-icon" href="themefile.tlx/1/06010205-1408612672/favicon.ico" />
    
    
    
            <title>NWL Media Server</title>
    </head><body class="widepage">
    <script type="text/javascript" src="j.tlx?v=06010205&amp;api=1.1&amp;themeid=1&amp;locale=en_GB"></script>
        <script type="text/javascript">
        IMS.objAPI = new IMS.API({slowCallCallback:function(){$("imsAjaxLoading")&&$("imsAjaxLoading").addClassName("visible")},slowCallFinishCallback:function(){$("imsAjaxLoading")&&$("imsAjaxLoading").removeClassName("visible");}});
        IU.observeRightClick(document, function(e){if(e.findElement("A")) return; e.stop();}, true);    </script>
    <div id="imsAjaxLoading">Processing...</div>
    <div id="imsHeader">
        <div id="imsLogo">
            <a class="bglogo" href="http://nwl.thirdlight.com/" title="NWL Media Server">&nbsp;</a>
            <div class="headerTop">
                <div id="loggedInDetails">
                                                                                                                                                                                                        <div id="emailFeedbackDetail" class="headerDetail"><a href="contact.tlx">Contact Us</a></div>                    <div id="loginDetail" class="headerDetail"><a href="login.tlx">Login &raquo;</a></div>                                    </div>
            </div>
        </div>
            
        <div class="headerBottom">
                            <div class="headerSearchContainer">
                    <form id="headerSearchForm" action="search.tlx" method="post">
                        <div>
                            <input id="headerSearchExactHidden" type="hidden" name="exact" value="0" />
                            <span class="beforetext"></span>&nbsp;<input type="text" class="searchInput headerSearch" name="formquery" value="Enter keywords..." />
                            <div class="headerSearchOptions">
                                <label><input type="checkbox" name="exactCheck" value="1"  onclick="$('headerSearchExactHidden').value=(this.checked?1:0);"/> Exact matches only</label>
                                                                <div class="submitwrapper">
                                                                    <button class="widget submit" type="submit">
    <img src="v6graphics/svg.tlx?img=tick" alt="" class="svg" />
<span class="label">Search</span>
</button>                                </div>
                            </div>
                        </div>
                    </form>
                </div>
                    </div>
        <div class="headerBottomShadow"></div>
    </div>
    <div id="imsBody">
        <script type="text/javascript">
/*<![CDATA[*/
UserAssetContextMenuOptions = [];
UserContainerContextMenuOptions = [];
UserCollectionContextMenuOptions = [];
UserSmartfolderContextMenuOptions = [];
UserAlbumContextMenuOptions = [];
AssetsCantBeRotated = {};
var thisItem = null;
        var AssetAvailability = {};
        
    function IsImageAvailable(nID) {
        if (AssetAvailability && AssetAvailability[nID]) {
            return false;
        }
        if ($('expiredbanner_' + nID)) {
            return false;
        }
        return true;
    }
    


            function CanEditAsset(nID) {
            return false;
        }
    

        function CanEditContainer(nID) {
    return false;
        }
    


        function IsAssetLinked(nID) {
    return false;
        }
        
    function IsImageDownloadWizardable(nID, elNode, enabledCallback) {
        IMS.objAPI.request("Downloads.CanDownloadWizard", {assetId:nID}, {onFailure:enabledCallback.curry(false), onSuccess:function(e){if (e.memo.response && e.memo.response.length) enabledCallback(true); else enabledCallback(false);}});
        return IsImageAvailable(nID);
    }
    function IsImageDownloadable(nID, elNode, enabledCallback) {
        IMS.objAPI.request("Downloads.CanDownloadOriginal", {assetId:nID}, {onFailure:enabledCallback.curry(false), onSuccess:function(e){if (e.memo.response && e.memo.response.length) enabledCallback(true); else enabledCallback(false);}});
        return IsImageAvailable(nID);
    }
    function IsImageDropboxExportable(nID, elNode, enabledCallback) {
        IMS.objAPI.request("Downloads.CanExportToDropbox", {assetId:nID}, {onFailure:enabledCallback.curry(false), onSuccess:function(e){if (e.memo.response && e.memo.response.length) enabledCallback(true); else enabledCallback(false);}});
        return IsImageAvailable(nID);
    }
    function IsImagePublishWebable(nID, elNode, enabledCallback) {
        IMS.objAPI.request("Downloads.CanPublishWeb", {assetId:nID}, {onFailure:enabledCallback.curry(false), onSuccess:function(e){if (e.memo.response && e.memo.response.canPublish) enabledCallback(true); else enabledCallback(false);}});
        return false;
    }
    
            UserAlbumContextMenuOptions = UserContainerContextMenuOptions.clone();
                        
    



                            

                        
                
        

        
        
            
EventMenuOptions = [];

UserMenuOptions = [];

function CanEditThisUser(uid) {
    return false;

}
function CanEditThisUserAccess(uid) {
    return false;

}
function CanDeleteThisUser(uid) {
    return false;

}
function AssetIDFromMouseEvent(e) {
    var target = e.target;
    var pattern = new RegExp('(^|\\s)contextMenu_Asset(\\s|$)');
    while(target && !pattern.test(target.className)) {
        target = target.parentNode;
    }
    if (target && pattern.test(target.className)) {
        var id = target.getAttribute("data-imsassetid");
        if (id) {
            AssetsCantBeRotated[id] = $(target).hasClassName("norotate");
            return id;
        }
    }
    return false;
}
function ContainerIDFromMouseEvent(e) {
    var target = e.target;
    while(target && !target.hasAttribute("data-imscontainerid")) {
        target = target.parentNode;
    }
    if(target) {
        var id = target.getAttribute("data-imscontainerid");
        if (id) return id;
    }
    return false;
}
function UserIDFromMouseEvent(e) {
    var target = e.target;
    var pattern = new RegExp('(^|\\s)contextMenu_User(\\s|$)');
    while(target && !pattern.test(target.className)) {
        target = target.parentNode;
    }
    if(target && pattern.test(target.className)) {
        var id = target.getAttribute("data-imsUserId");
        if (id) return id;
    }
    return false;
}
function EventIDFromMouseEvent(e) {
    var target = e.target;
    var pattern = new RegExp('(^|\\s)contextMenu_Event(\\s|$)');
    while(target && !pattern.test(target.className)) {
        target = target.parentNode;
    }
    if(target && pattern.test(target.className)) {
        var id = target.getAttribute("data-imsEventId");
        if (id) return id;
    }
    return false;
}
function GroupIDFromMouseEvent(e)
{
    var target = e.target;
    var pattern = new RegExp('(^|\\s)contextMenu_Group(\\s|$)');
    while(target && !pattern.test(target.className)) {
        target = target.parentNode;
    }
    if(target && pattern.test(target.className)) {
        var id = target.getAttribute("data-imsGroupId");
        if (id) return id;
    }
    return false;
}
IMS.contextMenus = {};
         IMS.contextMenus["assets"] = new Proto.Menu({
     selector: '.contextMenu_Asset',
     menuItems: UserAssetContextMenuOptions,
     idCallback: AssetIDFromMouseEvent,
     fade: true,
 extraArgs: ''
 });
         IMS.contextMenus["containers"] = new Proto.Menu({
     selector: '.contextMenu_Container',
     menuItems: UserContainerContextMenuOptions,
     idCallback: ContainerIDFromMouseEvent,
     fade: true,
 extraArgs: ''
 });
         IMS.contextMenus["collections"] = new Proto.Menu({
     selector: '.contextMenu_Collection',
     menuItems: UserCollectionContextMenuOptions,
     idCallback: ContainerIDFromMouseEvent,
     fade: true,
 extraArgs: ''
 });
         IMS.contextMenus["smartfolders"] = new Proto.Menu({
     selector: '.contextMenu_Smartfolder',
     menuItems: UserSmartfolderContextMenuOptions,
     idCallback: ContainerIDFromMouseEvent,
     fade: true,
 extraArgs: ''
 });
         IMS.contextMenus["albums"] = new Proto.Menu({
     selector: '.contextMenu_Album',
     menuItems: UserAlbumContextMenuOptions,
     idCallback: ContainerIDFromMouseEvent,
     fade: true,
 extraArgs: ''
 });

/*]]>*/
</script>        <div id="imsInnerBody">
            




                    







                
<div class="pageheading"><div class="headingbreadcrumb first" style="z-index:999"><a href="libraryhome.tlx">Home</a><div class="crumbbackgroundright"></div></div><div class="headingbreadcrumb last" style="z-index:99">Access Denied<div class="crumbbackgroundleft"></div><div class="crumbbackgroundright"></div></div><div class="clear">&nbsp;</div><div class="assetpagesheading"><img class="icon" src="v6graphics/thumbnails.icon.png" alt="" width="32" height="32"/><span class="title">Access Denied</span></div></div>
                <div class="tabbedPanelContainer">
    <div class="tabbedPanelInnerContainer">
                            <div class="tabbedPanelWide">
                                    <div class="tabbedPanel" id="contactformtabPanel">
                            
                                        <div class="optionpanelcontainer">
                    <div class="optionpanelrow note">
    <div class="notecontainer">Sorry, but you do not have access to this page.
</div>
</div>                <div style="clear:left;"></div>
</div>                                </div>
                            </div>
            </div>
            <div class="tabbedPanelButtonContainer wide narrowLeftPadding">
                              <span class="tabbedPanelButton">
                                                      <button type="button" class="widget button"  onclick="document.location.href=&#039;index.tlx&#039;;">
            <img src="v6graphics/svg.tlx?img=arrow" alt="" class="svg" />
            <span class="label">OK</span>
</button>                              </span>
                                <span class="tabbedPanelButton">
                                                      <button type="button" class="widget button"  onclick="document.location.href=&#039;login.tlx?forward=%2Fdofileupload.tlxr%3Ftask%3Dxb8xTf8WkMxGIHxIH.mex&#039;;">
            <img src="v6graphics/svg.tlx?img=arrow" alt="" class="svg" />
            <span class="label">Login</span>
</button>                              </span>
                      </div>
    </div>                </div>
    <div id="imsFooter" class="footerWidepage">Photographs copyright &copy; Northumbrian Water Ltd. <a href="http://www.thirdlight.com/">Digital Asset Management Software</a> by Third Light</div>
</div>




<script type="text/javascript">
/*<![CDATA[*/
        
    if (IMS.contextMenus && IMS.contextMenus.assets) IMS.contextMenus.assets.reattach();
    if (IMS.contextMenus && IMS.contextMenus.albums) IMS.contextMenus.albums.reattach();
    if (IMS.contextMenus && IMS.contextMenus.containers) IMS.contextMenus.containers.reattach();
    if (IMS.contextMenus && IMS.contextMenus.collections) IMS.contextMenus.collections.reattach();
    if (IMS.contextMenus && IMS.contextMenus.smartfolders) IMS.contextMenus.smartfolders.reattach();
/*]]>*/
</script>






<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www."); 
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));  
</script>
<script type="text/javascript">
var pageTracker = _gat._getTracker("40244447"); 
pageTracker._initData(); 
pageTracker._trackPageview();
</script>
</body>     
</html>

Am I missing a step or is the json request incorrect?

dominic

Third Light Staff

Registered: 2013-06-06

Posts: 119

Re: .Net Based File Upload

Hi Paul,

There are two distinct methods for uploading files from applications using the API:

- Upload.AddFilesToUpload is a regular API method, using the api.json.tlx endpoint - when doing this you should supply a hash including base64-encoded file data according to the documentation for that method.
- The POST URL returned from Upload.CreateUpload can be used as an alternative, and if so a file (and optionally metadata) should be POST-ed to the URL in multipart/form-data format. When calling from an application, it may be desirable to append "&t=ajax" to the POST URL - this will cause success/failure to be reported in JSON format, rather than as HTML documents or HTTP redirects.

The latter is preferred when possible, as it is considerably more efficient for large files.

Calls to other methods, including Upload.StartUpload, should always be made against the api.json.tlx endpoint.

Dominic

paulblighe

Member

Registered: 2014-11-11

Posts: 8

Re: .Net Based File Upload

Hi Dominic

Thanks for the explanation it has made it a bit clearer and I have managed to get the upload to work using the first option.  However, if I try to use the post url returned from CreateUpload I get the following error returned:

{"error":true,"message":"Sorry, but you do not have access to this page."}.  Is there any configuration I need to do to get access to the dofileupload.tlxr page?

Thanks
Paul

dominic

Third Light Staff

Registered: 2013-06-06

Posts: 119

Re: .Net Based File Upload

Hi Paul,

The things to check are that the "task" GET key is being passed through to the request correctly, and that either the 'allowotheruserupload' option was set in the call to Upload.CreateUpload or that the session key is sent (this can be done by sending an HTTP cookie called IMSSESSID). When 'allowotheruserupload' is not specified, IMS requires that the upload be performed in a session of the same user account used to create the upload task, hence the need to provide session details.

Dominic

paulblighe

Member

Registered: 2014-11-11

Posts: 8

Re: .Net Based File Upload

Hi Dominic

Thanks for the reply I have now got the upload working using the POST URL, this does not use the AddFilesToUpload method, is this correct?  Although I now have files in the system, I am now struggling to set the metadata for the file as part of the upload.  I am using a multipart/form-data httpwebrequest and am passing in the metadata as form data (the foreach on nvc is doing this. 

public static void HttpUploadFile(string urlToUse, string file, string sessionId, NameValueCollection nvc)
        {
            string boundary = "---------------------------" + DateTime.Now.Ticks.ToString("x");
            byte[] boundarybytes = System.Text.Encoding.ASCII.GetBytes("\r\n--" + boundary + "\r\n");

            HttpWebRequest wr = (HttpWebRequest)WebRequest.Create(urlToUse);
            wr.ContentType = "multipart/form-data; boundary=" + boundary;
            wr.Method = "POST";
            wr.KeepAlive = true;
            wr.Timeout = System.Threading.Timeout.Infinite;
            wr.Headers.Add("IMSSESSID", sessionId);

            Stream rs = wr.GetRequestStream();

            string formdataTemplate = "Content-Disposition: form-data; name=\"{0}\"\r\n\r\n{1}";
            foreach (string key in nvc.Keys)
            {
                rs.Write(boundarybytes, 0, boundarybytes.Length);
                string formitem = string.Format(formdataTemplate, key, nvc[key]);
                byte[] formitembytes = System.Text.Encoding.UTF8.GetBytes(formitem);
                rs.Write(formitembytes, 0, formitembytes.Length);
            }
            rs.Write(boundarybytes, 0, boundarybytes.Length);

            string fileUrl = file;
            string headerTemplate = "Content-Disposition: form-data; name=\"{0}\"; filename=\"{1}\"\r\nContent-Type: {2}\r\n\r\n";
            string header = string.Format(headerTemplate, "file", Path.GetFileName(fileUrl), Path.GetExtension(fileUrl));
            byte[] headerbytes = System.Text.Encoding.UTF8.GetBytes(header);
            rs.Write(headerbytes, 0, headerbytes.Length);

            FileStream fileStream = new FileStream(file, FileMode.Open, FileAccess.Read);
            byte[] buffer = new byte[4096];
            int bytesRead = 0;
            while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
            {
                rs.Write(buffer, 0, bytesRead);
            }
            fileStream.Close();

            byte[] trailer = System.Text.Encoding.ASCII.GetBytes("\r\n--" + boundary + "--\r\n");
            rs.Write(trailer, 0, trailer.Length);
            rs.Close();

            WebResponse wresp = null;
            try
            {
                wresp = wr.GetResponse();
                Stream stream2 = wresp.GetResponseStream();
                StreamReader reader2 = new StreamReader(stream2);
            }
            catch (Exception ex)
            {
                if (wresp != null)
                {
                    wresp.Close();
                    wresp = null;
                }
            }
            finally
            {
                wr = null;
            }
        }

I have assumed that the editablemetadata field in the CreateUpload function still needs to be set so I have set it in the request to CreateUpload.

string json = "{\"action\":\"Upload.CreateUpload\"," +
                               "\"apiVersion\":\"1.0\"," +
                               "\"inParams\":{\"params\":{\"destination\":\"" + parentId + "\"," +
                               "\"synchronous\":false," +
                               "\"allowotheruserupload\":true," +
                               "\"editablemetadata\":{\"cCTVClient\":\"OPTIONAL\",\"cCTVClientsJobReference\":\"OPTIONAL\",\"cCTVDepthatStartNode\":\"OPTIONAL\"," +
                               "\"cCTVFinishNodeReference\":\"OPTIONAL\",\"cCTVNameofSurveyor\":\"OPTIONAL\",\"cCTVPipelineLengthReference\":\"OPTIONAL\"," +
                               "\"cCTVStartNodeReference\":\"OPTIONAL\",\"cCTVLocationStreetName\":\"OPTIONAL\",\"cCTVLocationTownorVillage\":\"OPTIONAL\"}," +
                               "\"lifetime\":3600}" +
                               "}," +
                               "\"sessionId\":\"" + sessionId + "\"" +
                                "}";

When running the code no metadata is set.  Are there any eaxmples of passing metadata through a multipart form request?

Thanks
Paul

dominic

Third Light Staff

Registered: 2013-06-06

Posts: 119

Re: .Net Based File Upload

Hi Paul,

The POST upload target accepts metadata included as POST fields of the form "<file key>_<metadata field>" - in your posted code, file key looks to be set to "file", so these would become, for example "file_cCTVClient".

You are correct that the "editablemetadata" option to CreateUpload is still required, and similarly any defaults set through the "defaultmetadata" option are also supported. You are also correct that this replaces calling AddFilesToUpload.

Regards,

Dominic

paulblighe

Member

Registered: 2014-11-11

Posts: 8

Re: .Net Based File Upload

Hi Dominic

That's great.  I wasn't setting the name correctly on my NameValueCollection entries, changing it to <file key>_<metadatafield> has done the trick.  Thank you for your assistance on this.

For anyone else who is interested in this topic I will post example code once I have tidied it up and made sure everything is working correctly.

Thanks
Paul

paulblighe

Member

Registered: 2014-11-11

Posts: 8

Re: .Net Based File Upload

Hi Dominic

Is there anything I need to do in order to update a metadata field that is a date?  Do I have to pass it through in a certain format?

Thanks
Paul

paulblighe

Member

Registered: 2014-11-11

Posts: 8

Re: .Net Based File Upload

Ok the date format should be passed through as yyyy-MM-dd

paulblighe

Member

Registered: 2014-11-11

Posts: 8

Re: .Net Based File Upload

For anyone whos is looking to upload documents using .net this is based on the multipart form upload which does not use the Upload.AddFilesToUpload functionality.

Assuming you have a session id then the following steps should get you there:

Call Upload.CreateUpload which will return a post url and upload key (I have passed the parent folder id and session id as parameters and you will need to change the editablemetadata and add defaultmetadata if required):
               
               

var httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
                httpWebRequest.ContentType = "application/json";
                httpWebRequest.Method = "POST";

                using (var streamWriter1 = new StreamWriter(httpWebRequest.GetRequestStream()))
                {
                    string json = "{\"action\":\"Upload.CreateUpload\"," +
                                   "\"apiVersion\":\"1.0\"," +
                                   "\"inParams\":{\"params\":{\"destination\":\"" + parentId + "\"," +
                                   "\"synchronous\":false," +
                                   "\"allowotheruserupload\":true," +
                                   "\"editablemetadata\":{\"cCTVClient\":\"OPTIONAL\",\"cCTVClientsJobReference\":\"OPTIONAL\",\"cCTVDepthatStartNode\":\"OPTIONAL\"," +
                                   "\"cCTVFinishNodeReference\":\"OPTIONAL\",\"cCTVNameofSurveyor\":\"OPTIONAL\",\"cCTVPipelineLengthReference\":\"OPTIONAL\"," +
                                   "\"cCTVStartNodeReference\":\"OPTIONAL\",\"cCTVLocationStreetName\":\"OPTIONAL\",\"cCTVLocationTownorVillage\":\"OPTIONAL\"}," +
                                   "\"lifetime\":3600}" +
                                   "}," +
                                   "\"sessionId\":\"" + sessionId + "\"" +
                                    "}";

                    streamWriter1.Write(json);
                }
                var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
                using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
                {
                    returnValue = streamReader.ReadToEnd();
                }

Using the post url create the multipart/form post (if you want to pass metadata create this as a NameValueCollection, be aware that the key should be set to <file key>_<metadata> and that the metadata field should be transformed to camelcase e.g. CCTV_Client becomes cCTVClient.  The <file_key> value should be set to whatever you set the name of the file in the content-disposition, in terms of the code below I have set it to “file” therefore my NameValueCollection key woudl be file_cCTVClient.  You can set the file key to anything as long as you make sure that the namevaluecollection key contains it..  The url to post to should be set to the post url returned from CreateUpload and you will need to supply the session id in the header with the key as IMSSESSID:

public static string HttpUploadFile(string urlToUse, string file, string sessionId, NameValueCollection nvc)
        {
            string result = string.Empty;
            
            string boundary = "---------------------------" + DateTime.Now.Ticks.ToString("x");
            byte[] boundarybytes = System.Text.Encoding.ASCII.GetBytes("\r\n--" + boundary + "\r\n");

            HttpWebRequest wr = (HttpWebRequest)WebRequest.Create(urlToUse + "&t=ajax");
            wr.ContentType = "multipart/form-data; boundary=" + boundary;
            wr.Method = "POST";
            wr.KeepAlive = true;
            wr.Timeout = System.Threading.Timeout.Infinite;
            wr.Headers.Add("IMSSESSID", sessionId);

            Stream rs = wr.GetRequestStream();

            string formdataTemplate = "Content-Disposition: form-data; name=\"{0}\"\r\n\r\n{1}\r\n";
            foreach (string key in nvc.Keys)
            {
                rs.Write(boundarybytes, 0, boundarybytes.Length);
                string formitem = string.Format(formdataTemplate, key, nvc[key]);
                byte[] formitembytes = System.Text.Encoding.UTF8.GetBytes(formitem);
                rs.Write(formitembytes, 0, formitembytes.Length);
            }
            rs.Write(boundarybytes, 0, boundarybytes.Length);

            string fileUrl = file;
            string headerTemplate = "Content-Disposition: form-data; name=\"{0}\"; filename=\"{1}\"\r\nContent-Type: {2}\r\n\r\n";
            string header = string.Format(headerTemplate, "file", Path.GetFileName(fileUrl), Path.GetExtension(fileUrl));
            byte[] headerbytes = System.Text.Encoding.UTF8.GetBytes(header);
            rs.Write(headerbytes, 0, headerbytes.Length);

            FileStream fileStream = new FileStream(file, FileMode.Open, FileAccess.Read);
            byte[] buffer = new byte[4096];
            int bytesRead = 0;
            while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
            {
                rs.Write(buffer, 0, bytesRead);
            }
            fileStream.Close();

            byte[] trailer = System.Text.Encoding.ASCII.GetBytes("\r\n--" + boundary + "--\r\n");
            rs.Write(trailer, 0, trailer.Length);
            rs.Close();

            WebResponse wresp = null;
            try
            {
                wresp = wr.GetResponse();
                Stream stream2 = wresp.GetResponseStream();
                StreamReader reader2 = new StreamReader(stream2);
                result = reader2.ReadToEnd();

                reader2.Close();
                stream2.Close();
                wresp.Close();
            }
            catch (Exception ex)
            {
                if (wresp != null)
                {
                    wresp.Close();
                    wresp = null;
                }
            }
            finally
            {
                wr = null;
                
            }

            return result;
        }

Once you have successfully posted the file you need to call Upload.StartUpload passing in the session id and the upload key (from CreateUpload):

     

var httpWebRequest = (HttpWebRequest)WebRequest.Create(Helper.GetAppSettingAsString("APIUrl", false));
            httpWebRequest.ContentType = "application/json";
            httpWebRequest.Method = "POST";
            httpWebRequest.KeepAlive = true;
            httpWebRequest.Timeout = System.Threading.Timeout.Infinite;
            httpWebRequest.UserAgent = "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.70 Safari/533.4";

            using (var streamWriter1 = new StreamWriter(httpWebRequest.GetRequestStream()))
            {
                string json = "{\"action\":\"Upload.StartUpload\"," +
                               "\"apiVersion\":\"1.0\"," +
                               "\"sessionId\":\"" + sessionId + "\"," +
                               "\"inParams\":{\"uploadKey\":\"" + uploadKey + "\"" +
                               "}" +
                                "}";

                streamWriter1.Write(json);
            }
            var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
            string responseText;
            using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
            {
                responseText = streamReader.ReadToEnd();
            }

Then call Upload.GetUploadProgress passing in the session id and the upload key (from CreateUpload):

     

var httpWebRequest = (HttpWebRequest)WebRequest.Create(Helper.GetAppSettingAsString("APIUrl", false));
            httpWebRequest.ContentType = "application/json";
            httpWebRequest.Method = "POST";

            using (var streamWriter1 = new StreamWriter(httpWebRequest.GetRequestStream()))
            {
                string json = "{\"action\":\"Upload.GetUploadProgress\"," +
                               "\"apiVersion\":\"1.0\"," +
                               "\"sessionId\":\"" + sessionId + "\"," +
                               "\"inParams\":{\"uploadKey\":\"" + uploadKey + "\"" +
                               "}" +
                                "}";

                streamWriter1.Write(json);
            }
            var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
            using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
            {
                returnValue = streamReader.ReadToEnd();             
            }

At this point you will need to check the parameters coming back, you should get returned the Percent, NumberOfFiles, ProcessedFiles, RejectedFiles, uploadProgress.Params.Error. 

Once you get 100% then call Upload.FinaliseUpload passing in the upload key from CreateUpload:

     

var httpWebRequest = (HttpWebRequest)WebRequest.Create(Helper.GetAppSettingAsString("APIUrl", false));
            httpWebRequest.ContentType = "application/json";
            httpWebRequest.Method = "POST";

            using (var streamWriter1 = new StreamWriter(httpWebRequest.GetRequestStream()))
            {
                string json = "{\"action\":\"Upload.FinaliseUpload\"," +
                               "\"apiVersion\":\"1.0\"," +
                               "\"inParams\":{\"uploadKey\":\"" + uploadKey + "\"" +
                               "}" +
                                "}";

                streamWriter1.Write(json);
            }
            var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
            string responseText;
            using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
            {
                responseText = streamReader.ReadToEnd();                
            }

Then call Upload.CompleteUpload passing in the upload key from CreateUpload:

     

var httpWebRequest = (HttpWebRequest)WebRequest.Create(Helper.GetAppSettingAsString("APIUrl", false));
            httpWebRequest.ContentType = "application/json";
            httpWebRequest.Method = "POST";

            using (var streamWriter1 = new StreamWriter(httpWebRequest.GetRequestStream()))
            {
                string json = "{\"action\":\"Upload.CompleteUpload\"," +
                               "\"apiVersion\":\"1.0\"," +
                               "\"inParams\":{\"uploadKey\":\"" + uploadKey + "\"" +
                               "}" +
                                "}";

                streamWriter1.Write(json);
            }

            using (HttpWebResponse response = (HttpWebResponse)httpWebRequest.GetResponse())
            {
                string header = response.StatusCode.ToString();
            }

Hope this proves useful to anyone trying to do large file uploads via .net

Thanks
Paul