Friday, February 1, 2008

System Center Virtual Machine Manager file transfers causing disk fragmentation

I recently noticed a trend on the Microsoft Virtual Server hosts at the organization where I work. With no scheduled disk defragmentation taking place on the drives where the .VHD files are stored, fragmentation is rampant. Rampant is really an understatement, this fragmentation is quite extreme.

When first looking at the fragmentation of the host servers I figured a lack of a regular defrag schedule was the main problem....I am not so sure of that. Those of you familiar with and use Microsoft Virtual Machine Manager know that it moves files around by using BITs (Background Intelligent Transfer Service). BITs best feature is the ability to resume interrupted file transfers, which is very nice when migrating large Virtual Machines between hosts. BITs also seems to have another interesting perk:

Below is a simple test. I took a virtual host with no virtual machines. I ensured that the drive where the virtual machine would be placed was empty (no fragmentation).



I then deployed a virtual machine with one 16GB fixed size .VHD using SCVMM. Now a normal file copy of a 16GB file would result in a contiguous non-fragmented copy. After the VM was deployed by SCVMM, a disk defrag analyze resulted in the following report:

--------------------------------------------------------------------------------
Fragments File Size Most fragmented files
8,590 16.00 GB \VirtualServer\VirtualMachines\FRAG-TEST\FRAG-TEST_16GB-FIXED-W2K3-STD-x86-SP2.VHD


1 file deployed, 8,590 fragments?

So lets say you have a production host server that has no scheduled defragmentation. Day to day operations of deploying new machines and migrating existing machines is managed by SCVMM. What does the fragmentation look like after a few months of operation with about 25 virtual machines (with ALL FIXED SIZE .VHDs) running on it?

--------------------------------------------------------------------------------
Fragments File Size Most fragmented files
35,068 40.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
35,067 40.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
15,383 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
15,068 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
14,952 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
14,310 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
13,017 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
12,047 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
12,040 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
11,812 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
9,145 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
9,142 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
8,657 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
7,746 40.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
7,026 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
6,227 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
1,141 2.78 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
5 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
5 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
4 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
4 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
3 16.00 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 2.02 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 2.02 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 2.02 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 2.02 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 532 MB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 532 MB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 532 MB \VirtualServer\VirtualMachines\VHDFILE.vhd
2 2.02 GB \VirtualServer\VirtualMachines\VHDFILE.vhd
(VHD files renamed by author)

That is some serious fragmentation folks. I won't even throw in variables like dynamic disks, or checkpoints (which are always created as dynamic disks!). I'll buy a beverage of choice for the person who can calculate the number of extra disk I/O operations the storage array is performing because of this extreme fragmentation.

3 comments:

Anonymous said...

Hi Rawley,

I won't try to guess at the I/O, but I'll pass this issue along to our test team so it can be tracked and resolved.

Thanks,
James

Anonymous said...

" I'll buy a beverage of choice for the person who can calculate the number of extra disk I/O operations the storage array is performing because of this extreme fragmentation."

A lot. My beverage of choice is liquid gold. Pay up please :)

Anonymous said...

Hi Rawley,

We're having trouble reproducing the issue here. Could you give some more detail about the machine? We're seeing no more than 3 fragments with a BITS deployment. (We've not heard other reports of this issue.) Is it possible that the drive is under heavy load from another application?

Thanks,
James