HDInsight HBase: Migrating to new HDInsight version

Following are short steps to upgrade your HDInsight HBase cluster with small downtime.

Before you migrate please note that there may be incompatibilities between HBase Major/Minor version and below steps only works if there is no version compatibility issues between source and destination cluster. We recommend you to review HBase book before undertaking an upgrade.

Compatibility Type	Major	Minor	Patch
Client-Server wire Compatibility	N	Y	Y
Server-Server Compatibility	N	Y	Y
File Format Compatibility	N	Y	Y
Client API Compatibility	N	Y	Y
Client Binary Compatibility	N	N	Y
Server-Side Limited API Compatibility
Stable	N	Y	Y
Evolving	N	N	Y
Unstable	N	N	N
Dependency Compatibility	N	Y	Y
Operational Compatibility	N	N	Y

Scenario:

Migrate HDInsight 3.5 to 3.6 with same HBase major version or Migrate HDInsight between point versions e.g (3.6.2 to 3.6.3) of Azure HDInsight

Step1: Make sure that your application works with the new version.

Step2: Create HDInsight Cluster on same storage but with different container name

Step3: Flush your source HBase cluster https://raw.githubusercontent.com/Azure/hbase-utils/master/scripts/flush_all_tables.sh

Step4: Stop the ingestion to HBase cluster

Step5: Flush the cluster again with https://raw.githubusercontent.com/Azure/hbase-utils/master/scripts/flush_all_tables.sh [This will ensure that any remaining data in memstore is flushed]

Step6: Stop HBase services from Ambari

Step7: Go to new HDInsight cluster /Ambari HDFS settings and change the container name to original container –>Ambari –>HDFSàAdvanced –>Advanced core-site

Step8: Restart all required services [Ambari will indicate which services requires restart]

Step9: Point your application to the new cluster.

Step10: Start the ingestion to see if everything is working well

Step11: delete the original cluster after you see everything is working well.