shareVM- Share insights about using VM's

Simplify the use of virtualization in everyday life

Posts Tagged ‘ntfs

Best Practice: Defrag VMDK, VHD, VirtualBox Virtual Disk

leave a comment »

Wikipedia describes defragmentation as

a process that reduces the amount of fragmentation in file systems. It does this by physically organizing the contents of the disk to store the pieces of each file close together and contiguously. It also attempts to create larger regions of free space using compaction to impede the return of fragmentation.

Generically, the defragmentation of a Windows guest within a virtual disk running on a Windows host (Windows on Windows) requires a three-step process:

  1. Defragment the guest
  2. Defragment the virtual disk
  3. Defragment the host

On a Linux host or guest, the ext3 and ext4 file systems are more resilient to defragmentation.

Windows on Windows

You should perform the following steps whether you are using a Microsoft VHD, VirtualBox VDI or VMware VMDK virtual disk,

  1. On a Windows guest OS, run the Windows Disk Defragmenter to defragment the files within the volumes stored inside the virtual disk.
  2. Next, power down the virtual machine and defragment the virtual disk using contig. Defragmenting the virtual disk simply reorganizes the blocks so that used blocks move towards lower-numbered sectors and unused blocks move towards higher-numbered sectors.
  3. Run the Windows Disk Defragmenter to achieve an overall defragmentation of all files on the host including the virtual disk.

VMware VMDK specific

The following steps can be used generically for VMware VMDK, for Windows on WIndows or any other suppoted platforms. vmware-vdiskmanger:is a standalone tool for defragmenting a growable VMware Workstation, VMware Fusion or VMware Server, vmdk when it is offline. Note that you cannot defragment:

  • Preallocated virtual disks
  • Physical hard drives
  • Virtual disks that are associated with snapshots.

The recommended steps for defragmenting a vmdk are:

  1. On a Windows guest OS, run the Windows Disk Defragmenter to defragment the files within the volumes stored inside the VMDK.
  2. Next, power down the virtual machine and defragment the vmdk using the command vmware-vdiskmanager -d myVirtualDisk.vmdk. Defragmenting the vmdk simply reorganizes the blocks so that used blocks move towards lower-numbered sectors and unused blocks move towards higher-numbered sectors.
  3. If the host OS is also Windows, run the Windows Disk Defragmenter to achieve an overall defragmentation of all files on the host including the VMDK.

Tool to find out how much will vmdk, vhd shrink on Windows, Ubuntu

leave a comment »

Now you can download and run vmsi to determine whether it is worth shrinking that VM on your disk. We are releasing vmsi for both the Windows NTFS and Ubuntu platforms. Please read the Release Notes to understand the current limitations.

Let us know how it works, please send email to  support at sharevm dot com

Written by paule1s

January 12, 2009 at 12:56 am

How much will the VM shrink?

with one comment

Shrinking a VM today is a time consuming process: zero out the free blocks, defrag, use vmware’s tools to shrink it partition by parition. I wanted a tool which would tell me after doing all that how much would it help. This turned out to be a complicated process as you have to find the free clusters in NTFS and find out whether they are actually allocated as in sparse VMDK files all blocks may not be allocated. I ran the tool on some VMs see below: Windows 2003, Windows NT & a couple of XP VMs.

fcp -f c:\work\vhd\w2k3.vhd
NTFS Free Sectors 594632
Free Sectors Allocated in Virtual Image 294496
Maximum Possible Saving by Shrinking 143 MB

fcp -f “c:\work\vm\w1\Windows XP Professional.vmdk”
NTFS Free Sectors 1049160
Free Sectors Allocated in Virtual Image 883144
Maximum Possible Saving by Shrinking 431 MB

fcp -f “c:\work\vhd\Windows XP Hard Disk.vhd”
NTFS Free Sectors 30124176
Free Sectors Allocated in Virtual Image 847013
Maximum Possible Saving by Shrinking 413 MB

fcp -f “c:\work\vm\wnta\wnta.vmdk”
NTFS Free Sectors 27724512
Free Sectors Allocated in Virtual Image 71656
Maximum Possible Saving by Shrinking 34 MB

I am thinking of releasing the tool, it only supports vmdk and vhd files today with NTFS file system. If you would like to get an early release, would be thrilled to share let me know.

Written by RS

December 31, 2008 at 8:21 am

Virtual Machine Disk Image Compression

with 2 comments

Experience with running out of storage

Background about sparse files, Windows NTFS and VHD

VHD Pre-Compactor & Compactor

When you create a new VM you can pre-allocate all the storage in one shot or let the storage grow on demand. In the latter case, the VM uses a sparse file

VMDK Wipe & Shrink

3rd Party (Not from MSFT & VMW)

Rsync: for transferring files between machines

Research

Analysis of Sparse VMDK File

leave a comment »

I did some analysis of the layout of a VMDK file. The graph below shows this analysis: x-axis is the distance in the vmdk file for two logically adjacent blocks(sectors). For for example is block 33 is at offset 3467 and  block 34 is at offset 3468 then the distance in the vmdk file is 3468-3467=1.  The Y-axis is log to the base 2 of the frequency of the distance.

Distance (in VMDK file) of logically adjacent blocks

The graph shows that most logically adjacent blocks have adjacent offsets in the vmdk file. This file had a grain size of 128, so you would expect that only 2/128 sectors would have non-adjacent neighbours (1/64). That is roughly the case. The rest of the distances are clustered around multiples of 128 (as one would expect?). I am not sure if this has to do with NTFS allocating blocks in some sequence or VMW workstation expanding the disk in a particular way.

The vmdk was a windows vmdk 4GB disk with windows XP. Very little use after installing the base OS. Installed a couple of packages like Flex Builder.

Another interesting plot would be to plot the actual distance on the physical disk on the host. Thats next.

Written by RS

December 4, 2008 at 2:46 pm

Compressing Virtual Images

leave a comment »

Experience with running out of storage

Background about sparse files, Windows NTFS and VHD

VHD Pre-Compactor & Compactor

When you create a new VM you can pre-allocate all the storage in one shot or let the storage grow on demand. In the latter case, the VM uses a sparse file

VMDK Wipe & Shrink

3rd Party (Not from MSFT & VMW)

Rsync: for transferring files between machines

Research