Hyper-V Concepts - Snapshots
This is an article with content gleaned from Hyper-V forum threads - you could consider the input here based on true stories.
The intent of the article is to explain the concepts of snapshots specific to Hyper-V. The reason for posting the thread is that most of the key concepts are covered as well as specific scenarios provided from real world trials.
To view the architectural components using the online viewer for the Windows Server 2008 R2 Hyper-V Component Architecture Poster, click the image.
An often encountered issue is that Windows Server 2008 administrator starts to get low on disk space, and so begins deleting unneeded snapshots. Using the Hyper-V manager console, they delete the snapshot subtree.
However, the avhd file is still in the snapshot folder and they get the error "the file is used for another program and can not delete". Then they shut down the VM and can see that the merge progress going, but they get impatient and and turn off the VM.
This and other Hyper-V shapshot issues are discussed in this video: Ben Armstrong on Hyper-V Snapshot Common Issues
This blog article explains Confusion over Hyper-V and Snapshots: http://www.aidanfinn.com/?p=11935
You must wait for the merge process to finish. Then you will be able to delete the .avhd file.
So the FAQ is: what is the right way to delete un-needed snapshots?
The process is:
- Delete the snapshot
- Power Off the VM
- Wait patiently for the AVHD to merge with the VHD (depending on the snapshot file size, this might take a long time)
- Power On the VM.
NOTE:
- After the merge is complete the AVHD is deleted by the system.
- The VM must be powered off for the merge of the AVHD to the VHD to happen.
- Manually interrupting this process can corrupt your VHD - Hyper-V has checks in place to 'do no harm' if the merge is not successful and leave everything as it was.
- The more constrained for disk space that your system is, the longer the
- process takes.
- There is no way to predict how long the merge will take.
See also:
http://itproctology.blogspot.com/2008/06/hyper-v-lost-my-vm-and-i-connected-tot.html
http://itproctology.blogspot.com/2009/04/never-resize-vhd-with-snapshots-or.html
http://itproctology.blogspot.com/2009/04/hyper-v-snapshots-are-tkaing-up-large.html
Cause: - the pause critical state means that you don't even have enough space to run the VM. Temporarily attach a drive to the host and Export the VM to
that drive - you can use a USB drive if necessary. After the export is finished, you can delete the original (then delete the VHD files of the original).
Then copy the folder that was created by the export process back to the host storage and Import.
VM -> snapshot -> install patch, test patch, approve patch -> Delete snapshot
-> power off VM -> wait -> power on VM.
VM -> snapshot -> install patch, test patch, patch is not approved -> revert
(technically you are still running as a snapshot)
The AVHD is not merged with the VHD while the VM is running - the application in the VM must suffer downtime for the merge to happen after a snapshot is
deleted.The hypervisor actually knows that it has to perform a merge, and it is waiting for the VM to be powered off in order to perform it. If the VM is never powered off, the merge never begins.
In the UI you see that the snapshot is gone - really only the meta information of the snapshot is gone - the disk state of the snapshot continues, until the VM is powered off.
The merge only happens if the snapshots are deleted. If the snapshots are not deleted, they remain no matter the state of the VM.
Checkpoint = SCVMM term (VMware Virtual Center term also)
Snapshot = Hyper-V Term
In the MSFT world, these terms are interchangeable. Do not confuse this with a VMWare checkpoint - that is a different thing (the use case is similar but the implementation is very different)
Both SCVMM and Hyper-V manager offer snapshots/checkpoints for the same use cases (the terms are 100% interchangeable). If the Integration Components are installed in the VM (and running) then a running snapshot of hte VM is allowed.You can always take an offline (non-running) shapshot. SCVMM does apply some rules that the Hyper-V manager does not. Other than that SCVMM simply uses the Hyper-V technologies to perform the exact same actions.
Q: If we keep the snapshots (one per month), will it affect the live migration?
A: This is a design issue. If you are running Hyper-V R2 then all of your snapshots and VHDs are stored together, thus allowing migration to happen in
all cases.
Q: If you have over 10 snapshots, will it affect VM performance?
A: It can. Most folks avoid this (and it is recommended to) fo a VM that is running in production as it introduces a potential performance hit, but also
additional IO overhead on the disk system (a read can be spread across all disks in the chain, not just the latest - a write affects only the latest -
thus extra work for reads as there are more files involved)
Q: Will it merge take longer since it has 10 snapshots?
A: Most likely. The reason is that each disk is merged one by one into its parent, until the last merge happens.
If your requirement is zero downtime, then I (very frankly) recommend using a more traditional backup solution and not using snapshots. Or using SAN level back-up, snapshot, etc. tools (sheer speed).
Actually, the running VM has the latest state(I created some new files) when I delete the snapshots even through the snapshots are not there. If I reboot the VM with snapshots deleted, VM has all my latest created files. Just wonder how do these latest changes keep (in memory?) even if the snapshots are not there? If I do not shut down the VM to let merge process to begin, how long can I keep this latest state even though the snapshots file are not there ?
This latest state exists in a differencing disk. The .AVHD that you see in the storage location of the VM. when you create a snapshot a differencing disk is spawned. And gives you the ability to return to a point in time (the time the snapshot was taken) by simply deleting the latest differencing disk and making a new one. When you delete a snapshot - it disappears from the UI - all the physical bits of the snapshot still exist, it is only not displayed in the management console - but all the pieces still exist until the merge is complete.
http://itproctology.blogspot.com/2008/03/snapshottingnot-your-checkpointing.html
Q: Only recently have I discovered the evil of running snapshots in a production environment. I have inherited a 2008 Server running one VHD which has one active snapshot which all my users are connecting to. The AVHD files have grown large one is 200Gb the other is 250Gb. I have removed all the older snapshots by deleting them from the UI and brought down the server last night to allow the merge. After 7 hours the merge reached 25% and I unfortunately had to bring the server backup as my users were now jumping up and down.
My issue and question is that I can only get approx 8 hours downtime for the merge. Can I run these merges in 8hr increments starting the server each time. My ultimate goal here is to get back to one AVHD file and eventually merge this with the VHD and get rid of snapshots all together.
A: The best option would have been to delete one snapshot at a time.
Power off the VM, delete one snapshot, allow it to merge, assess the time involved, make a judgement call, move to the next or wait until the next window.
What is happening in your current state is that during the power off - the system is merging all of the snapshots together, one at a time (the total is shown in the UI, but it is many steps, once for each AVHD).
If one snapshot is fully merged, then space is freed and the process moves to the next. If you need to pow on the VM an a merge has not finished, it is essentially undone and you continue in your current state - until you can provide a longer window. Since the merge is incremental it goes one snapshot at a time, it is just that there has to be enough time for one to complete before the next can begin - the order cannot be changed.
If you are running Server 2008 R2 there is an option (this is specific to Server 2008 R2 and requires free local storage) - you can power off the VM, create a new snapshot and then you can export the snapshot (not the 'now' but the snapshot).
This will create an export that contains a single VHD.
You can then Import this Export of your VM and be back to a single VHD with no snapshots. Verify it. Delete the original from the UI (then manually delete the VHD - BTW when you delete the VM from the UI, it will sit and background merge all of the snapshots into a single VHD - you have to wait for this to complete before you can delete the VHD without error).