A problem that soon arises is - what about performance? The currently obvious solution is to add paravirtualization features like VMWare VMI and Xen dom0 to both the guest kernel and (with somewhat less intrusiveness) to the host kernel. Paravirtualized virtual machines don't pretend they're accessing actual hardware but call into the VM host environment for common functions. This also simplifies the VM host as it doesn't have to emulate hardware (like PCI access) but can use a more efficient protocol for the purpose.
What about all the efforts invested in modern kernel infrastructure and algorithms at the guest side? Does it mean that it's completely irrelevant what kind of CPU scheduler is implemented in the guest since the code is eventually rescheduled in the VM host OS (probably yes)? Does it mean file system journalling is useless on the guest side if it's properly implemented (full data journalling) on the host (probably yes)? Does it mean memory is wasted on disk caching in the guest VM (maybe yes, maybe no, since the local kernel knows how to track which local file blocks are accessed and why)? How much performance could be recovered by simply using the dumbest algorithms in the guest kernel?
The question is - where does this stop? Do we have a function that reads a block from an (emulated) drive and returns it? A function that passes packets in memory from the host to the guest? Replace the interrupt table and the concept of interrupts with a table of fuction pointers for callbacks (ala method dispatch in OOP)? At the extreme, does a wholly paravirtualized kernel reduce to a microkernel passing messages between the userland and the hypervisor?
At the end, what really matters beyond a certain point is the userland - how consistent it is (a plus for FreeBSD) and how many applications are available (plus for Linux and Windows).
#1 Right on
I like the ideas about schedulers and file system journalling (although most big deployments using VMs the VMs themselves use direct fiber or iscsi). Maybe things will go that way soon also.
#2 Need for heterogenous OS VM support
One could argue that freebsd jails are the first best lightest form of VM out there (as good as solaris zones in many ways) ... with virtnet and quota and vfs improvments going in the kernel FreeBSD will soon really rock in terms of "virtualization" support ... for itself. I think this excellent support has helped and hindered FreeBSD. Since one machine can host 100s of jailed "instances" of FreeBSD and this has long been seen as "good enough" - besides who'd run anything but *more* FreeBSD if given the choice.
What's needed today is support for FreeBSD *in* virtualized hosting environments (on VMWare, Xen, VirtualBox) but also support for running those sorts of virtualization technologues *on* FreeBSD.
Support for FreeBSD *in* VMs seems ok-ish (certainly weaker than Linux) ... FreeBSD in VirtualBox doesn't run too well and as I recall VMware doesn't officially support FreeBSD but it mostly works ... FreeBSD in Xen is coming along.
Hosting VM technologies to run other OSes *on* FreeBSD is pretty much non-existent: I guess it's not virtualization but I have qemu running in production ;-) The great hope here is for Xen!