Azure storage block blob upload from Android
Android rules the smartphone Apps world, according to AppBrain there are 1,272,196 apps in Android marketplace by June 26, 2014. Microsoft "Azure Mobile service has SDKs to support Windows" Phone, iOS, "*Android and HTML developers on top of powerful REST APIs. enabling developers to build connected applications across a varied range of platforms and provide a consistent experience across devices. These services can enable a user to toggle between devices and continue where he/she last left off, enabling seamless device transitions.
Mobile Services enable application developers to:
*
- *Store Data in the cloud *
- Authenticate users easily
- Easily send push notification
- Easy customization to enable use of selected services
- Support for monitoring, alerting and auto-scaling" [Vipul Patel on codeguru.com]
"In the most general sense, the term "blob" is commonly understood to mean "Binary Large OBject." Many of us are familiar with this term from its usage in database-land, where "blob data" might be data stored in our database which does not conform to an established data type as defined by the database. Such data are usually (if the database supports it) persisted as plain binary data (image files come to mind as an example)." [John Atten on codeproject.com]
"According to the Azure team, the most common use-cases for blob storage will involve Block Blobs. Block blobs represent binary data that has been segmented into one or more blocks to enable ease of transmission over a network, and sensible management of large data files. The blocks which make up a blob may be of different sizes, up to 4 MB each. Each block within a blob is identified by a Block ID, and may optionally also include an MD5 hash of the blob content. The maximum size for a block blob is 200 GB, and a blob can consist of up to 50,000 individual blocks." [John Atten on codeproject.com]
Most developers use Azure Blob storage to upload their BLOB data from their apps, and Android developers can use the following code to upload a BLOB from the device.
// upload file to azure blob storage
private static Boolean upload(String sasUrl, String filePath, String mimeType) {
try {
// Get the file data
File file = new File(filePath);
if (!file.exists()) {
return false;
}
String absoluteFilePath = file.getAbsolutePath();
FileInputStream fis = new FileInputStream(absoluteFilePath);
int bytesRead = 0;
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] b = new byte[1024];
while ((bytesRead = fis.read(b)) != -1) {
bos.write(b, 0, bytesRead);
}
fis.close();
byte[] bytes = bos.toByteArray();
// Post our image data (byte array) to the server
URL url = new URL(sasUrl.replace("\"", ""));
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setDoOutput(true);
urlConnection.setConnectTimeout(15000);
urlConnection.setReadTimeout(15000);
urlConnection.setRequestMethod("PUT");
urlConnection.addRequestProperty("Content-Type", mimeType);
urlConnection.setRequestProperty("Content-Length", "" + bytes.length);
urlConnection.setRequestProperty("x-ms-blob-type", "BlockBlob");
// Write file data to server
DataOutputStream wr = new DataOutputStream(urlConnection.getOutputStream());
wr.write(bytes);
wr.flush();
wr.close();
int response = urlConnection.getResponseCode();
if (response == 201 && urlConnection.getResponseMessage().equals("Created")) {
return true;
}
} catch (Exception e) {
e.printStackTrace();
}
return false;
}
Please note that sasUrl parameter below is a signed URL acquired from the web service.
The code is working fine for small blobs but when a blob reaches a certain size depending on the phone we are testing with, we start to get out of memory exceptions. We would like to split the blobs and upload them into blocks. However, most examples available on the web are C# based and are using the Storage Client library.
For a solution to the problem, there is an Azure Storage Android library published here. A basic can be given as (ref: https://raw.githubusercontent.com/Azure/azure-storage-android/master/microsoft-azure-storage-samples/src/com/microsoft/azure/storage/blob/gettingstarted/BlobBasics.java)
/** * Copyright Microsoft Corporation * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */package com.microsoft.azure.storage.blob.gettingstarted; import java.net.URISyntaxException; import java.security.InvalidKeyException; import com.microsoft.azure.storage.CloudStorageAccount; import com.microsoft.azure.storage.blob.BlobContainerPermissions; import com.microsoft.azure.storage.blob.BlobContainerPublicAccessType; import com.microsoft.azure.storage.blob.CloudBlobClient; import com.microsoft.azure.storage.blob.CloudBlobContainer; import com.microsoft.azure.storage.blob.CloudBlockBlob; import com.microsoft.azure.storage.blob.ListBlobItem; import com.microsoft.azure.storage.util.Utility; /** * This sample illustrates basic usage of the various Blob Primitives provided * in the Storage Client Library including CloudBlobContainer, CloudBlockBlob * and CloudBlobClient. */public class BlobBasics { /** * Executes the sample. * * @param args * No input args are expected from users. * @throws URISyntaxException * @throws InvalidKeyException */ public static void main(String[] args) throws InvalidKeyException, URISyntaxException { Utility.printSampleStartInfo("BlobBasics"); // Setup the cloud storage account. CloudStorageAccount account = CloudStorageAccount.parse(Utility.storageConnectionString); // Create a blob service client CloudBlobClient blobClient = account.createCloudBlobClient(); try { // Get a reference to a container // The container name must be lower case CloudBlobContainer container = blobClient.getContainerReference("blobbasicscontainer"); // Create the container if it does not exist container.createIfNotExists(); // Make the container public // Create a permissions object BlobContainerPermissions containerPermissions = new BlobContainerPermissions(); // Include public access in the permissions object containerPermissions.setPublicAccess(BlobContainerPublicAccessType.CONTAINER); // Set the permissions on the container container.uploadPermissions(containerPermissions); // Upload 3 blobs // Get a reference to a blob in the container CloudBlockBlob blob1 = container.getBlockBlobReference("blobbasicsblob1"); // Upload text to the blob blob1.uploadText("Hello, World1"); // Get a reference to a blob in the container CloudBlockBlob blob2 = container.getBlockBlobReference("blobbasicsblob2"); // Upload text to the blob blob2.uploadText("Hello, World2"); // Get a reference to a blob in the container CloudBlockBlob blob3 = container.getBlockBlobReference("blobbasicsblob3"); // Upload text to the blob blob3.uploadText("Hello, World3"); // Download the blob // For each item in the container for (ListBlobItem blobItem : container.listBlobs()) { // If the item is a blob, not a virtual directory if (blobItem instanceof CloudBlockBlob) { // Download the text CloudBlockBlob retrievedBlob = (CloudBlockBlob) blobItem; System.out.println(retrievedBlob.downloadText()); } } // List the blobs in a container, loop over them and // output the URI of each of them for (ListBlobItem blobItem : container.listBlobs()) { System.out.println(blobItem.getUri()); } // Delete the blobs blob1.deleteIfExists(); blob2.deleteIfExists(); blob3.deleteIfExists(); // Delete the container container.deleteIfExists(); } catch (Throwable t) { Utility.printException(t); } Utility.printSampleCompleteInfo("BlobBasics"); } }
"The method" for the solution *"to use is uploadFromFile in the blob class. This will, by default attempt to put the blob in a single put if the size is less than 64MB and otherwise send the blob in 4MB blocks. If you’d like to reduce the 64MB limit, you can set the singleBlobPutThresholdInBytes property on the BlobRequestOptions object of either the CloudBlobClient (which will affect all requests) or passed to the uploadFromFile method (to affect only that request). The storage library includes many convenient features such as automated retries and maximum execution timeout across the block put requests which are all configurable.
If you’d still like to use a more manual approach, the PutBlock and Put Block List API references are here and provide generic, cross-language documentation. These have nice wrappers in the CloudBlockBlob class of the Azure Storage Android library called uploadBlock and commitBlockList which may save you a lot of time in manual request construction and can provide some of the aforementioned conveniences."* [Emily Gerner (Microsoft) on stackoverflow.com]