Share via

Migrate Sharepoint document with version to Sharepoint online using CSOM

  • Article

It has been observed quite a few time that METALOGIX failed to migrate large document library content with version enabled or library using managed metadata. Very often it gets timed out exception during the operation.

To overcome this issue in one of our migration project we used CSOM to migrate the documents with versions.

Using the following code piece we can migrate the documents with their version from one platform to the other one.

 

1. Prepare the source and target client context here I am assuming target is sharepoint online

  ServicePointManager.ServerCertificateValidationCallback += customXertificateValidation;
 ClientContext sourceContext = new ClientContext(sourceSiteUrl);
 sourceContext.Credentials = new NetworkCredential(sourceUserName, sourcePassWord, sourceDomain);

 
 ClientContext targetContext = new ClientContext(targetSiteUrl);
 var passWord = new SecureString();
 foreach (char c in targetPassWord.ToCharArray()) passWord.AppendChar(c);
 targetContext.Credentials = new SharePointOnlineCredentials(targetUserName, passWord);

 2. Get the source folder

  Folder sourceFolder = sourceContext.Web.GetFolderByServerRelativeUrl(sourceFolderUrl); 
 sourceContext.Load(sourceFolder, fldr => fldr.Name);
 sourceContext.Load(sourceFolder.Files, files => files.Include(
 file => file.ListItemAllFields
 ));

 try
 {
 sourceContext.ExecuteQuery();
 }
 catch (Exception e)
 {
              
 }

 3. Get the target folder

  Folder targetSiteFolder = targetContext.Web.GetFolderByServerRelativeUrl(targetFolderUrl);
 targetContext.Load(targetSiteFolder.Files, files => files.Include(
 file => file.ListItemAllFields
 ));
 targetContext.Load(targetSiteFolder);

 try
 {
 targetContext.ExecuteQuery();
 }
 catch (Exception ex)
 {
                
 }

 4. Do the migration operation

  int fileCount = sourceFolder.Files.Count();

 for (int i = 0; i < fileCount; i++)
 {
 Microsoft.SharePoint.Client.File sourceFile = sourceFolder.Files[i];

 FileVersionCollection sourceFileVersions = sFile.Versions; sContext.Load(sourceFileVersions, fileVersions => fileVersions.Include( version => version.Url, version => version.VersionLabel, version => version.CheckInComment )); try { sContext.ExecuteQuery(); } catch (Exception e) { //handle exception   

  }

 foreach (FileVersion fileVer in sourceFileVersions)

 { UploadFiles(sContext.Url + "/" + fileVer.Url, targetFolder, sourceItem["FileLeafRef"].ToString());

 }

 }

 5.

 private static void UploadFiles(string sourceFileUrl, Folder targetFolder, string fileName)

  {

  FileCreationInformation targetFileVersionCreationInfo = new FileCreationInformation(); targetFileVersionCreationInfo.Overwrite = true; try { WebRequest request = HttpWebRequest.Create(sourceFileUrl); request.Credentials = sContext.Credentials; using (WebResponse response = request.GetResponse()) { using (Stream stream = response.GetResponseStream()) { byte[] verBuffer = new byte[32768]; using (MemoryStream versionMS = new MemoryStream()) { int read; while ((read = stream.Read(verBuffer, 0, verBuffer.Length)) > 0) { versionMS.Write(verBuffer, 0, read); } versionMS.Seek(0, SeekOrigin.Begin); targetFileVersionCreationInfo.ContentStream = versionMS; tContext.RequestTimeout = System.Threading.Timeout.Infinite; targetFileVersionCreationInfo.Url = targetFolder.ServerRelativeUrl + "/" + fileName; Microsoft.SharePoint.Client.File targetVersionFile = targetFolder.Files.Add(targetFileVersionCreationInfo); try { tContext.ExecuteQuery(); } catch (Exception ex) { //handle exception   

  } } } } } catch (Exception ex) { //handle exception   

  }

  }

  

 The above piece of code is able to migrate the document with version but it has few issues 

 Issue 1. The created date,modified date,credated by and modified by field will be changed to keep these field intact we need to do the following

  List list = targetContext.Web.Lists.GetByTitle(fldr.Name);
 ListItem item = file.ListItemAllFields;
 targetContext.Load(list, listObject => listObject.EnableVersioning, listObject => listObject.EnableMinorVersions);
 targetContext.Load(item, itemObject => itemObject.File);
 targetContext.ExecuteQuery(); 
 [PLACEHOLDER 1] 
 targetContext.ExecuteQuery();
 FieldUserValue targetCreatedUser = new FieldUserValue();
 targetCreatedUser.LookupId = createdUser.Id;
 FieldUserValue targetModifiedUser = new FieldUserValue();
 targetModifiedUser.LookupId = modifieddUser.Id; 
 item["Editor"] = targetModifiedUser;
 item["Author"] = targetCreatedUser;
 item["Created"] = created;
 item["Modified"] = modified;


 item.Update(); 
 [PLACEHOLDER 2] 

  targetContext.ExecuteQuery();

 Issue 2: In attempt to keep the created and modified information intact the code internally create additional version as the library is enabled with version changes.

 To resolvethat we need to put the following code in PLACEHOLDER1 and PLACEHOLDER 2 subsequently

 PLACEHOLDER1

  var isVersionEnable = list.EnableVersioning;
 list.EnableVersioning = false; 

  list.Update(); 

 PLACEHOLDER 2 

  list.EnableVersioning = true; 
 list.Update(); 

  

 Issue 3:  With the above fixes we are done with migrating documents with versions for the libraries which are version enabled and Forcecheckout checkin option set as false.

  If the library is configured for Forcechecckout then with the above piece of code execution we will be end up with documents with latest version all the previous version info will not be available. To overcome this issue we need to do a small hack before staring actual migration code.

 We need to set the forcecheckout option for the target library as false.

   targetList.ForceCheckout = false;

 and at the end of migrating all the doc just set it back with actual settings.

Comments

  • Anonymous
    February 11, 2016
    Great article! Thanks for sharing! One issue I encountered is WebRequest request = HttpWebRequest.Create(sourceFileUrl); request.Credentials = sContext.Credentials It throws 403 error. I have to use something like                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(sourceFileUrl);                request.Credentials = sContext.Credentials;                request.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f");                request.Method = "GET";

  • Anonymous
    February 21, 2016
    Another issue is if there is more than 5000 subfolders, the code will fail silently (0 subfolder returned). CAML query will need to be used

  • Anonymous
    March 13, 2016
    How do I run these commands/scripts from powershell?