Node.js script for releasing a Windows Azure blob lease
This post covers a workaround for an issue that may affect you if you’re deploying Windows Azure virtual machines from VHDs stored in Windows Azure blob storage. The issue doesn’t always occur (in fact, our team hasn’t been able to repro it), and it will be fixed soon. If you run into the issue, you can use any one of several workarounds covered below.
Blob leases are a mechanism provided by Windows Azure for ensuring that only one process has write access to a blob. As Steve Marx notes in his blog post on the topic, “A lease is the distributed equivalent of a lock. Locks are rarely, if ever, used in distributed systems, because when components or networks fail in a distributed system, it’s easy to leave the entire system in a deadlock situation. Leases alleviate that problem, since there’s a built-in timeout, after which resources will be accessible again.”
In the case of VHD images stored as blobs, Windows Azure uses a lease to ensure that only one virtual machine at a time has the VHD mounted in a read/write configuration. In certain cases, however, we’ve found that the lease may not expire correctly after deleting the virtual machine and deleting the disk or OS image associated with the VHD. This can cause a lease conflict error message to occur when you try to delete the VHD or re-use it later in a different virtual machine.
If you’re affected by this issue, you can explicitly break the lease that has not expired, or you can make a copy of the VHD and use that copy for provisioning a new virtual machine. Craig Landis has posted instructions on the Windows Azure community forum for how to do this from Windows machines; he also covers the same techniques in a separate post addressing a variation on the issue.
For those who are managing Windows Azure virtual machines from Linux or Mac desktops, our team has developed a Node.js script that can be used to break a lease if needed. Here are the steps to follow for installing and running the script:
1. Verify through the Windows Azure management portal that the VHD is not actually in use. Craig’s forum post provides guidance on how to do this.
2. If you don’t have the Windows Azure command line tool for Mac and Linux installed, you can get it by installing the Windows Azure SDK for Node.js. SDK installation instructions for Windows, Mac, and Linux can be found on the Windows Azure Node.js Developer Center.
3. Download and import your Windows Azure publish settings file, as covered under “Manage your account information and publish settings” in the command line tool documentation.
4. Copy the the breakLease.js file (available here) to the node_modules/azure-cli subfolder under your Node.js global modules folder. You can find your global modules folder with the npm ls –g command. For example, on my Windows machine that command returns c:\Users\dmahugh\AppData\Roaming\npm, so I need to copy the script to c:\Users\dmahugh\AppData\Roaming\npm\node_modules\azure-cli.
After you’ve completed those setup steps, you can break a blob lease by running the script with a single parameter, the URL of the blob:
> node breakLease.js <absolute-url-to-blob>
The script prints out information about the steps it takes to break the lease:
That’s all there is to it. As I mentioned earlier, this workaround is only needed in certain cases until the underlying cause has been fixed. Please let us know if you run into any issues using this script.
Comments
- Anonymous
August 29, 2012
I've added the code to a Gist on GitHub to make it easier for people to copy and potentially contribute against. The code can be found here: gist.github.com/3526416