Need to move ~50TB of data from an Azure VM to an Azure File share in the same tenant

Question

I need to move 50tb from an Azure VM file share to Azure files. All of the Azure data move solutions are primarily targeted for on-premises to Azure, but I see almost no options for an Azure VM to Azure Files. Azure storage mover agent are image files for Hyper-v or VMware only so I can't use that. Azure file sync is intended for on-premises to Azure, but even so I don't think it'd be a help to me in this scenario.

After a lot of searching and failed attempts, I'm left with the solution of AzCopy/Azure storage mover to move the data. With there being ~50tb this will take a long time. Is there another solution I'm missing that could handle this bulk move better than AzCopy/Azure storage mover?

Answer 1

Hi @Weinberg, Eric

Welcome to Microsoft Q&A Forum. Thanks for posting your query.

Yes, there are couple of other solutions which you can try to move bulk data. There are multiple tools available which helps in moving the data from Azure VM to Azure File share.

I would recommend you use the AzCopy, Robocopy and Azure Data Factory based on your scenario.

Actually, AzCopy is also a powerful tool for transferring large amounts of data and can be optimized for better performance, use --parallel-level option with a higher value which will increase the number of concurrent operations and adjust the --block-size option to higher MB. Also use a --recursive flag to copy files.

Example:-

azcopy copy "https://<your-vm-file-share-url>" "https://<your-storage-account>.file.core.windows.net/<your-file-share>" --recursive --parallel-level=64 --block-size=8M

Recommended	Tool	Support for Azure file shares	Preservation of file fidelity
	Azure Storage Mover	Supported.	Full fidelity.*
	Azure Storage Mover	Supported.	Full fidelity.*
	Azure Storage Mover	Supported.	Full fidelity.*
	RoboCopy	Supported. Azure file shares can be mounted as network drives.	Full fidelity.*
	Azure File Sync	Natively integrated into Azure file shares.	Full fidelity.*
	Azure Storage Migration Program	Supported.	Full fidelity.*
	Storage Migration Service	Indirectly supported. Azure file shares can be mounted as network drives on SMS target servers.	Full fidelity.*
	Data Box (including the data copy service to load files onto the device)	Supported. (Data Box Disks doesn't support large file shares)	Data Box and Data Box Heavy fully support metadata. Data Box Disks does not preserve file metadata.
	AzCopy latest version	Supported but not fully recommended.	Doesn't support differential copies at scale, and some file fidelity might be lost. Learn how to use AzCopy with Azure file shares
	Azure Storage Explorer latest version	Supported but not recommended.	Loses most file fidelity, like ACLs. Supports timestamps.
	Azure Data Factory	Supported.	Doesn't copy metadata.

If your Azure VM is running windows, you can use RoboCopy to transfer files to your Azure file share. It is suitable for large data transfers and can be run from your Azure VM to copy files to the Azure File share efficiently.

If you Azure VM and Azure file share are in same region, Mount both the source (Azure VM file share) and target (Azure File share) to the VM using the storage account key.

Using the following command to perform the copy operation:

robocopy <source> <target> /MIR /COPYALL /MT:16 /R:2 /W:1 /B /IT /DCOPY:DAT

Replace <source> and <target> with the appropriate paths.

After the initial copy, you can run the command again to capture any changes made during the copy.

However, you can also leverage using the ADF, you can create a pipeline that copies data from your Azure VM (using a self-hosted integration runtime) to Azure Files. This may require some setup and configuration. This can also be a good way to manage the transfer.

Please refer for additional information:

• Migrate to SMB Azure file shares
• Azure solution for data transfer

I hope this helps in resolving the query.

Please let us know if you have any further queries, we will be glad to assist you closely.

Please do consider to “up-vote” wherever the information provided helps you, this can be beneficial to other community members.