Updating Metadata during Copy Blob and Snapshot Blob
This issue has been resolved for copy blob operations in the Windows Azure SDK 1.3 release which can be downloaded here . The snapshot issue will be resolved in a future release.
When you copy or snapshot a blob in the Windows Azure Blob Service, you can specify the metadata to be stored on the snapshot or the destination blob. If no metadata is provided in the request, the server will copy the metadata from the source blob (the blob to copy from or to snapshot). However, if metadata is provided, the service will not copy the metadata from the source blob and just store the new metadata provided. Additionally, you cannot change the metadata of a snapshot after it has been created.
The current Storage Client Library in the Windows Azure SDK allows you to specify the metadata to send for CopyBlob, but there is an issue that causes the updated metadata not actually to be sent to the server. This can cause the application to incorrectly believe that it has updated the metadata on CopyBlob when it actually hasn’t been updated.
Until this is fixed in a future version of the SDK, you will need to use your own extension method if you want to update the metadata during CopyBlob. The extension method will take in metadata to send to the server. Then if the application wants to add more metadata to the existing metadata of the blob being copied, the application would then first fetch metadata on the source blob and then send all the metadata including the new ones to the extension CopyBlob operation.
In addition, the SnapshotBlob operation in the SDK does not allow sending metadata. For SnapshotBlob, we gave an extension method that does this in our post on protecting blobs from application errors. This can be easily extended for “CopyBlob” too and we have the code at the end of this post, which can be copied into the same extension class.
We now explain the issue with CopyBlob using some code examples. If you want to update the metadata of the destination of the CopyBlob, you would want to set the metadata on the destination blob instance before invoking CopyBlob, as shown below. In the current SDK when doing this, the metadata is not sent to the server. This results in the server copying all the metadata from the source to destination. But on the client end, the destination instance continues to have the new metadata set by the application. The following code shows the problem:
CloudBlob destinationBlob = cloudContainer.GetBlobReference("mydocs/PDC09_draft2.ppt");
// set metadata “version” on destinationBlob so that once copied, the destinationBlob instance
// should only have “version” as the metadata
destinationBlob.Attributes.Metadata.Add("version", "draft2");
// BUG: CopyBlob does not send “version” in the REST protocol so server goes ahead and copies
// any metadata present in the sourceBlob
destinationBlob.CopyFromBlob(sourceBlob);
Solution: The solution is to use the CopyBlob extension method at the end of this post as follows:
CloudBlob destinationBlob = cloudContainer.GetBlobReference("mydocs/PDC09_draft2.ppt");
// Get the metadata from source blob if you want to copy them too to the destination blob and add
// the new version metadata
NameValueCollection metadata = new NameValueCollection();
metadata.Add(sourceBlob.Metadata);
metadata.Add("version", "draft2");
BlobRequestOptions options = new BlobRequestOptions()
{
Timeout = TimeSpan.FromSeconds(45),
RetryPolicy = RetryPolicies.RetryExponential(
RetryPolicies.DefaultClientRetryCount, RetryPolicies.DefaultClientBackoff)
};
// Send the metadata too with the copy operation
destinationBlob.CopyBlob(
sourceBlob,
metadata,
Microsoft.WindowsAzure.StorageClient.Protocol.ConditionHeaderKind.None,
null /*sourceConditionValues*/,
null /*leaseId*/,
options);
Another issue to be aware of is that if a CopyFromBlob operation results in the metadata being copied from source to destination, an application developer may expect the destination blob instance to have the metadata set. However, since the copy operation does not return the metadata, the operation does not set the metadata on the destination blob instance in your application. The application will therefore need to call FetchAttributes explicitly to retrieve the metadata from the server as shown in the following:
// Let us assume sourceBlob already has metadata “author” set and we are just copying the blob
// Let us assume sourceBlob already has metadata “author” set and we are just copying the blob
CloudBlob destinationBlob = cloudContainer.GetBlobReference("mydocs/PDC09_draft3.ppt");
destinationBlob.CopyFromBlob(sourceBlob);
// before FetchAttributes is called, destinationBlob instance has no metadata even though the
// server has copied metadata from the source blob.
destinationBlob.FetchAttributes();
// Now destinationBlob instance has the metadata “author” available.
Jai Haridas
Here is the code for Copy Blob extension method.
/// <summary>
/// Copy blob with new metadata
/// </summary>
public static CloudBlob CopyBlob(
this CloudBlob destinationBlob,
CloudBlob sourceBlob,
NameValueCollection destinationBlobMetadata,
ConditionHeaderKind sourceConditions,
string sourceConditionValues,
string leaseId,
BlobRequestOptions options)
{
if (sourceBlob == null)
{
throw new ArgumentNullException("sourceBlob");
}
if (destinationBlob == null)
{
throw new ArgumentNullException("destinationBlob");
}
ShouldRetry shouldRetry = options.RetryPolicy == null ?
RetryPolicies.RetryExponential(RetryPolicies.DefaultClientRetryCount,
RetryPolicies.DefaultClientBackoff)() : options.RetryPolicy();
int currentRetryCount = -1;
for (; ; )
{
currentRetryCount++;
try
{
TimeSpan timeout = options.Timeout.HasValue ? options.Timeout.Value : TimeSpan.FromSeconds(30);
return BlobExtensions.CopyBlob(
destinationBlob,
sourceBlob,
destinationBlobMetadata,
timeout,
sourceConditions,
sourceConditionValues,
leaseId);
}
catch (InvalidOperationException e)
{
// TODO: Log the exception here for debugging
// Check if we need to retry using the required policy
TimeSpan delay;
if (!IsExceptionRetryable(e) || !shouldRetry(currentRetryCount, e, out delay))
{
throw;
}
System.Threading.Thread.Sleep((int)delay.TotalMilliseconds);
}
}
}
private static CloudBlob CopyBlob(
this CloudBlob destinationBlob,
CloudBlob sourceBlob,
NameValueCollection destinationBlobMetadata,
TimeSpan timeout,
ConditionHeaderKind sourceConditions,
string sourceConditionValues,
string leaseId)
{
StringBuilder canonicalName = new StringBuilder();
canonicalName.AppendFormat(
CultureInfo.InvariantCulture,
"/{0}{1}",
sourceBlob.ServiceClient.Credentials.AccountName,
sourceBlob.Uri.AbsolutePath);
if (sourceBlob.SnapshotTime.HasValue)
{
canonicalName.AppendFormat("?snapshot={0}", sourceBlob.SnapshotTime.Value.ToString(
"yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fffffff'Z'", CultureInfo.InvariantCulture));
}
HttpWebRequest request = BlobRequest.CopyFrom(
destinationBlob.Uri,
(int)timeout.TotalSeconds,
canonicalName.ToString(),
null ,
sourceConditions,
sourceConditionValues,
leaseId);
// Adding 2 seconds to have the timeouton webrequest slightly more than the timeout we set for the server to
// complete the operation
request.Timeout = (int)timeout.TotalMilliseconds + 2000;
if (destinationBlobMetadata != null)
{
foreach (string key in destinationBlobMetadata.Keys)
{
request.Headers.Add("x-ms-meta-" + key, destinationBlobMetadata[key]);
}
}
destinationBlob.ServiceClient.Credentials.SignRequest(request);
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
destinationBlob.FetchAttributes();
return destinationBlob;
}
}