Freigeben über

AzCopy – Transfer data with re-startable mode and SAS Token

  • Artikel

Recently, we released a new version of AzCopy with a set of new features.

You can always download latest version from aka.ms/AzCopy.

New features added

  • /DestSAS and /SourceSAS: This option allows access to storage containers and blobs with a SAS (Shared Access Signature) token. SAS token, which is generated by the storage account owner, grants access to specific containers and blobs with specifc permissions and for a specified period of time. Permissions include LIST, READ, WRITE or DELETE.
    Currently, we support SAS token while you upload data to Azure Storage or download data from Azure Storage. For blob copy, we support SAS Token for the source location.
    Please refer to this blog post for more information on How to create a Shared Access Signature.
  • Enhancement of re-startable mode: In previous version, the re-startable mode is supported for interruption caused by network or other issues during file transfer but it only supports restarting the transfer from the beginning of the interrupted file. Imagine if the interrupted file is a large file (e.g. a VHD file) and most part of it has already been transferred to Azure Storage, it would be time-consuming to restart the transfer of entire file again. So to improve this scenario, we enhanced re-startable mode and made it restart the transfer from the point of interruption. For block blob, we chose 4Mb as the size of data chunk.

 

Example1: Upload all files from a local directory to a container using SAS token which offers permits for list and write

AzCopy C:\blobData https://xyzaccount.blob.core.windows.net/xyzcontainer /DestSAS:”?sr=c&si=mypolicy&sig=XXXXX” /s

/DestSAS here is for you to specify the SAS token to access storage container, it should be enclosed in quotes.

 

Example2: Download all blobs from a container to a local directory and then delete them from source (by using /Mov). The SAS token offers list, read and delete permission.

AzCopy https://xyzaccount.blob.core.windows.net/xyzcontainer C:\blobData /SourceSAS:”?sr=c&si=mypolicy&sig=XXXXX” /MOV /s

 

Example3: Upload all files from a local directory to a container in re-startable mode.

AzCopy C:\blobData https://myaccount.blob.core.windows.net/mycontainer /destkey:key /Z:restart.log /s

/Z is the parameter to turn on the re-startable mode. ‘restart.log’ is the customer-defined name of the journal file which will be saved in local.

For instance, if “C:\blobData” folder contains four files below and the size of file2.vhd is 1GB.

C:\blobData\file1.docx

C:\blobData\file2.vhd

C:\blobData\file3.txt

C:\blobData\file4.txt

Assume an interrupt occurs while copying the “file2.vhd”, and 90% of the file has been transferred to Azure already. Then upon restarting, we will only transfer the remaining 10% of “file2.vhd” and another two remaining files.

 

Jason Tang

Comments

  • Anonymous
    February 13, 2014
    Where does the file actually get installed to? I've ran the .msi but can't find azcopy...

  • Anonymous
    February 14, 2014
    Please check C:Program Files (x86)Microsoft SDKsWindows AzureAzCopy

  • Anonymous
    February 15, 2014
    Hi Jason, Thanks a lot. Tnis "Please check C:Program Files (x86)Microsoft SDKsWindows AzureAzCopy" should be inside the article.

  • Anonymous
    March 02, 2014
    Why are we getting errors such as "The remote server returned an error: (403) Forbidden" I've used the code to generate the policy, and still receive an error.  it works fine with the SourceKey parm.  Also tried different variants of the Storage Library, including the 2.1 version as some posts on Stack Overflow indicate. msdn.microsoft.com/.../jj721951.aspx Here's the command: AzCopy zzz.blob.core.windows.net/vhds zzz.blob.core.windows.net/vhds /SourceSAS:"?sv=2012-02-12&sr=c&si=unpolicy&sig=kdTPj7BTVtvTUqqbWUAfNCwSFLgr5hgQuF9FluPU%2F5k%3D" /DestKey:"Zg6yIu0dreMpIUiJTnaJbHShBuY6riYYYZ8ckOZ5D5iS35FOXrLzM/tfMf5587B9/u+NufFfl9bsTicMhy3HJw==" /Z:AzCopy-journal.log /V:AzCopy-verbose.log diab4n13.v32201402281848330226.vhd

  • Anonymous
    March 04, 2014
    The comment has been removed

  • Anonymous
    March 04, 2014
    The comment has been removed

  • Anonymous
    March 21, 2014
    How do I download only files created in the blob since a specific date so I can limit the download to only recent files added to the blob?