System Center Virtual Machine Manager file transfers causing disk fragmentation

I recently noticed a trend on the Microsoft Virtual Server hosts at the organization where I work. With no scheduled disk defragmentation taking place on the drives where the .VHD files are stored, fragmentation is rampant. Rampant is really an understatement, this fragmentation is quite extreme.

When first looking at the fragmentation of the host servers I figured a lack of a regular defrag schedule was the main problem....I am not so sure of that. Those of you familiar with and use Microsoft Virtual Machine Manager know that it moves files around by using BITs (Background Intelligent Transfer Service). BITs best feature is the ability to resume interrupted file transfers, which is very nice when migrating large Virtual Machines between hosts. BITs also seems to have another interesting perk:

Below is a simple test. I took a virtual host with no virtual machines. I ensured that the drive where the virtual machine would be placed was empty (no fragmentation).



I then deployed a virtual machine with one 16GB fixed size .VHD using SCVMM. Now a normal file copy of a 16GB file would result in a contiguous non-fragmented copy. After the VM was deployed by SCVMM, a disk defrag analyze resulted in the following report:

--------------------------------------------------------------------------------
Fragments File Size Most fragmented files
8,590 16.00 GB \VirtualServer\VirtualMachines\FRAG-TEST\FRAG-TEST_16GB-FIXED-W2K3-STD-x86-SP2.VHD

1 file deployed, 8,590 fragments?

So lets say you have a production host server that has no scheduled defragmentation. Day to day operations of deploying new machines and migrating existing machines is managed by SCVMM. What does the fragmentation look like after a few months of operation with about 25 virtual machines (with ALL FIXED SIZE .VHDs) running on it?

--------------------------------------------------------------------------------
Fragments File Size Most fragmented files
35,068 40.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
35,067 40.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
15,383 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
15,068 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
14,952 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
14,310 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
13,017 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
12,047 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
12,040 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
11,812 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
9,145 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
9,142 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
8,657 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
7,746 40.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
7,026 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
6,227 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
1,141 2.78 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
5 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
5 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
4 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
4 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
3 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 2.02 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 2.02 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 2.02 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 2.02 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 532 MB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 532 MB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 532 MB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 2.02 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
(VHD files renamed by author)

That is some serious fragmentation folks. I won't even throw in variables like dynamic disks, or checkpoints (which are always created as dynamic disks!). I'll buy a beverage of choice for the person who can calculate the number of extra disk I/O operations the storage array is performing because of this extreme fragmentation.

System Center Virtual Machine Manager (SCVMM) Remote Library Management

When we first implemented SCVMM into our organization one challenge that was recognized was how to easily manage our remote library shares. With the potential for several remote libraries, it immediately became cumbersome to ensure that each location had all of the new or updated system images.


We immediately began to work on a system to allow us to manage 1 local library share, and have the contents of that mirrored at each remote location. Windows 2003 R2 has a new feature that fit the role perfectly.


Enter Server 2003 R2 DFSR :

For those not familiar with DFSR, it offers a load of potential configurations and uses. The automatic replication and the compression built into it were the main benefits for the library replication.

The compression method used by DFSR is called Remote Differential Compression (RDC), more Here. A cool thing about RDC is that it is optimized for VHD files. A quick scenario of RDC is as follows:

VHD A is a Windows 2003 Server Standard Edition with SP1 and is 5GB

VHD B is a Windows 2003 Server Standard Edition with SP2 and is 5.2GB

RDC breaks the VHDs down into smaller parts and replicates, then the parts are put back together at the remote location. VHD A is replicated first. Once replication of VHD B begins, each part of the file is compared to other file parts that already exist at the remote location. Parts that match are not replicated, but reused from parts existing at the remote site.

This means that VHD A has all 5GB replicated. But only the different parts of VHD B have to be replicated. So we'll say the .2GB is the only data that has to be replicated for VHD B!

Since the majority of our SCVMM library content is VHDs and ISO files, DFSR really yields itself as the perfect companion. We've seen as much as 70-80% reduction in WAN traffic from using DFSR. The main benefit that I like though is I can update an image and place it in our local library, and I know that it will also be available at our remote sites once it is replicated.

Here is a quick diagram of the setup I described above: