how “capitalism” forces virtualization downstream
Over the last decade we have systematically added a layer of indirection at every interface in the stack. These days we call this virtualization!
On a NetApp Filer we had
> raid disk group > volume
The problem was you could not expand/shrink Raid Groups on the fly, you couldn’t move data easily between different Raid Groups. We get a layer of indirection or virtualization
> raid disk group > aggregate > flex volume>
Since an aggregate was logical instead of physical, it could be expanded or shrunk without changing the volume, you could move data around.
On a USB Disk
If we look inside the disk itself, especially usb flash devices we went from
cylinder, heads, sector > logical table > device abstraction
Again this allowed the rotation of different logical sectors to different physical cells, to ensure a single cell was not rewritten more times than its lifetime.
In a SAN
we put a switch between the Raid Groups and the computer. The switch puts a layer of indirection between the blocks and the computer
You knew all that :-). So what does it have to do with capitalism. My simplistic definition of capitalism is that the system will remove all inefficiencies in a chain and who ever will remove them stands to benefit economically. Or said another way: money finds its way into the right pockets!
So look at the stack today:
chips > motherboard, network, storage, bios > hypervisor > OS > Security, Backup etc > Business, Productivity Apps
Every layer presents an interface to the layer above. Each layer is also owned by different companies in the eco-system. Each of those companies has pressure to maximize its revenue. Tasked with this difficult challenge, you look at the layer above and see what is selling and can you add it to your layer. Happens naturally over time: intel added virtulization support, phoenix bios is adding the hypervisor, operating systems are trying to add backup and security …. The cycle goes on ….
Virtualization will be “innovated” always in a higher layer of the stack and commoditized by the lower layers.
The higher layer in the stack finds a lot of new functionality and benefit by making interface to a lower layer “logical”. They take this to market, till at some point the lower layer realizes that this is their API, they should move virtualization into their layer. The pressure to do this is extreme and the time frame to monetize this really small:
- Imagine the tussle between VMW and the storage vendors. VMW introduces logical disks with cloning, but storage vendors want to offer logical luns and volumes and disk files, as this moves the cloning functionality from the hypervisor to the storage.
- Imagine: Western Digital or Seagate could create multiple disks (vhd/vmdk files) on a single physical disk and then offer the capabilities to grow, shrink, move data between them. Even add networking to the disk controller, then different disks can connect to each other. They can do that if the processing power, memory reach a price point that it can be embedded directly into the component or lower layer. Which is what effectively happened to computing.
- VMW introduces logical network switch, Cisco jumps in with nexus-V
For a consumer this is a good thing, but money and value are shifting down stack across different companies, which have to co-exist in the eco-system (cisco, intel, emc, vmw), yet guard their innovation from becoming commoditized.