Cloud-to-Cloud: Migrating Ubuntu Linux Virtual Machines to Microsoft Azure from Amazon AWS
A short time ago, I assisted an organization with defining a process for migrating web applications and data running on Ubuntu Linux virtual machines to the Microsoft Azure cloud platform from Amazon AWS. In this article, I’m documenting the general steps we used in an attempt to help other organizations that may be planning a similar migration approach.
NOTE: These steps were performed to migrate virtual machines running applications on Ubuntu Linux 14.04 and Apache2 to Microsoft Azure. The specific steps presented in this article may vary based on Linux operating system version as well as installed applications and packages. Use at your own discretion.
Task 1: Before you begin … BACKUP!
The migration process presented in this article makes minimal changes to the source virtual machines to prepare them for a successful migration. However, as with all migrations, performing and verifying a complete system backup is recommended to ensure that you can easily revert back to the original application environment, if necessary.
Task 2: Confirm source OS distro and version
For the migration to be successful, the same Linux operating system distribution and version should be used for both source and target virtual machines. Others have successfully migrated between particular distros or versions, but the migration process for such a migration often requires an intimate knowledge of the source and target operating system configurations and can involve several additional tasks that are unique to each operating system version in use.
On each source VM, you can confirm the operating system distro and version with the following commands:
cat /etc/issue
uname --all
Task 3: Provision target virtual machines on Microsoft Azure
Based on the information collected above in Task 2, provision new target virtual machines on the Microsoft Azure cloud platform using the same operating system distribution and version as the source virtual machines. I've recently published a Quick Start Guide which can be used to step through this process.
Task 4: Update source and target virtual machines to latest package versions
To help ensure that platform images between cloud providers are running the same version of key operating system packages, update these packages to the latest versions on both source and target virtual machines.
sudo apt-get update
sudo apt-get upgrade
Task 5: Install rsync and screen packages on source and target virtual machines
The migration of application packages and files in this process will use rsync over ssh between source and target virtual machines. The actual transfer of files between virtual machines can take some time, so I also recommend using screen so that you can easily re-attach to an in-progress migration session if you are inadvertently disconnected.
Ensure that rsync and screen packages are installed on both the source and target virtual machines with these commands:
sudo apt-get install rsync
sudo apt-get install screen
Task 6: Add a consistent user account to both source and target virtual machines
To facilitate the migration process, ensure that you have a consistent user account configured on both source and target virtual machines with sudo enabled. The newly provisioned target virtual machines from Task 3 already include a user named azureuser with sudo enabled. To configure this same user on each source virtual machine, use the following commands:
sudo groupadd -g 500 azureuser
sudo useradd -u 500 -g 500 -m -s /bin/bash azureuser
sudo passwd azureuser
When prompted for a new password for azureuser, enter the same password used when provisioning the target virtual machines in Task 3.
Task 7: Start a screen session for the migration
On the source virtual machine, enter a new screen session for the migration by using the following command:
sudo screen -S AzureMigration
If you are disconnected from the source virtual machine during the migration process, you can reconnect to the detached screen session by using the following command after signing in again to the source virtual machine:
sudo screen -r
Task 8: Build an exclusion list of directories and files
During the migration, we want to be careful to skip any files that include configuration information relating to the identity of the source virtual machines, such as IP addresses, hostnames, ssh keys, etc. For the Ubuntu-based virtual machines that we migrated, we used the following commands on each source virtual machine to build our list of directories and files to exclude from the migration process:
EXCLUDEFILE=/tmp/exclude.file
EXCLUDELIST='/boot /etc/fstab /etc/hostname /etc/issue /etc/hosts
/etc/sudoers /etc/networks /etc/network/* /etc/resolv.conf
/etc/ssh/* /etc/sysctl.conf /etc/mtab /etc/udev/rules.d/*
/lock /net /tmp'EXCLUDEPATH=$(echo $EXCLUDELIST | sed 's/\ /\\n/g')
echo -e $EXCLUDEPATH > $EXCLUDEFILE
find / -name '*cloud-init*' >> $EXCLUDEFILE
find / -name '*cloud-config*' >> $EXCLUDEFILE
find / -name '*cloud-final*' >> $EXCLUDEFILE
The actual list of directories and files that you exclude may vary from this list, based on the Linux distro version, packages and applications that you are migrating.
Credits: Kudos to Kevin Carter who wrote a great article a couple years ago that provided a useful starting point for building a list of directories and files to consider excluding as part of a Linux-to-Linux migration process!
Task 9: Stop applications during migration
To minimize application data changes from occurring during the migration process, stop the related applications and daemons on the source virtual machines. The application that we migrated was a web application built using Apache2, so we simply stopped the related Apache2 daemon.
sudo service stop apache2
Task 10: Migrate the application files and data
From each source virtual machine, migrate application files and data using two rsync passes over ssh. The first pass performs the bulk of the data transfer, whereas the second pass uses checksums to confirm that all files were transferred successfully.
TARGETVM="insert_target_vm_public_ip_address"
rsync -e "ssh" -rlpEAXogDtSzh -P -x --exclude-from="$EXCLUDEFILE" --rsync-path="sudo rsync" --verbose --progress / azureuser@$TARGETVM:/
rsync -e "ssh" -crlpEAXogDtSzh -P -x --exclude-from="$EXCLUDEFILE" --rsync-path="sudo rsync" --verbose --progress / azureuser@$TARGETVM:/
Task 11: Restart each target virtual machine
After both rsync passes have completed, restart each target virtual machine to complete the migration process.
ssh azureuser@$TARGETVM
shutdown -r now
Completed! What's your migration scenario?
Do you have a different migration scenario from what's been presented in this article? If so, feel free to leave the details for your scenario in the Comments section below for us to consider documenting in a future article.