HDInsight:- Attach additional Azure storage accounts to the cluster

This blog is discontinued in favor of updated HDInsight documentation on MSDN /en-us/azure/hdinsight/hdinsight-hadoop-add-storage

 

HDInsight supports a notion of the default file system. The default file system implies a default scheme and authority. It can also be used to resolve relative paths. During the HDInsight creation process, an Azure Storage account and a specific Azure Blob storage container from that account is designated as the default file system.

In addition to this storage account, you can add additional storage accounts from the same Azure subscription or different Azure subscriptions during the creation process or after a cluster has been provisioned

Attach additional accounts during cluster provisioning

This bit is easy, while creating the HDInsight cluster go to optional configuration and add additional storage accounts

Attach additional account to existing cluster

Storage accounts to the existing clusters can be added via Ambari by following below steps

Step 1:
Go to Ambari dashboard https://.azurehdinsight.net/

Step 2:
Navigate to HDFS -->Config -->Advanced, scroll down to Custom core-site

.                         

Step 3:-
Select Add Property and enter your storage account name and key in following manner

Key: fs.azure.account.key..blob.core.windows.net
Value: yourkey

Step 4: Restart services from Ambari

Step 5: Test if you are able to access the data

xxxyy@hn0-clustername:~$ hadoop fs -ls wasbs://containername@ashishhbasestorageadl.blob.core.windows.net/folder

This is a cross post: originally posted at https://blogs.msdn.microsoft.com/ashish/2016/08/25/hdinsight-attach-additional-azure-storage-accounts/ Follow @ashishth