Some virtualization benchmarks

I did some benchmarks of the three currently most prominent virtualization platforms: VMWare ESX 3.5 U3, Citrix XenServer 5.0 U2, Microsoft Hyper-V 2008 R2 on three 64-bit guest operating systems: Ubuntu 8.10, FreeBSD 7.1 and Windows Server 2008 R2 beta. The results are mostly better then I thought they will be. Especially suprising was FreeBSD's more than decent performance which actually lead the others in one benchmark.

The benchmark was serving PHP pages with Apache, with PHP as FastCGI on Linux and FreeBSD and as mod_php on Windows. There are 3x3 combinations, though I couldn't get Windows 2008 to work as a guest under Xen, so instead there's Linux in paravirtualization (PV) mode. All other combinations use full virtualization, backed by CPU extensions. The host was 2x4-core Xeon 5405 @ 2 GHz and 16 GB RAM, but the guests were always configured with only 4 CPUs and 4 GB RAM. Of course, the machines were configured and benchmarked separately, with only one VM running at any one time.

The PHP test application consisted of some 50,000 lines with 5 include files and outputed content between 1 kB and 5 kB. No PHP accelerators and caches were used. Benchmarks were done with siege by passing it a list of URLs (a population of 5000 references to input files processed by the script, very reproducible). The number of simultaneous HTTP clients ("users") was varied. Runs were always done with warm file system caches. The client was connected by giagbit Etheret via a patch cable to the server.

Apache was always threaded - the worker MPM was used on Linux and FreeBSD and the native thread model on Windows. HTTP logging was disabled.

I can't really explain it, but the wholly-virtualized FreeBSD machine under ESXi was consistently almost as fast as the paravirtualized Xen Linux on this benchmark. Other benchmarks I did did not show such close performance, but FreeBSD still did very good. For example, a similar static HTML page benchmark showed similar trends, with Xen-Linux-PV peaking at 7400 TPS and Xen-FreeBSD peaking at 5600 TPS, (two best results of that test) between 10 and 20 users, but both settled at 5100 TPS with 50 users. Disk performance was decent, but network performance was bad on anything but VMWare.

Since ESXi is now free, it was a clear winner of the test.

#1 Re: Some virtualization benchmarks

Added on 2009-04-26T12:31 by Romain Tartière

Regarding FreeBSD configuration, was the accf_http module loaded or built in the kernel?

#2 Re: Some virtualization benchmarks

Added on 2009-04-28T11:07 by

XenServer is free, too. And seems to be more feature-rich than ESXi.

#3 Re: Some virtualization benchmarks

Added on 2009-04-28T11:20 by Ivan Voras

@1: accf_http was not used, vanilla configuration

@2: I also tried XenServer, and it was very dissapointing, from performance that was noticably worse, to the surprisingly slow console, unusuable over ADSL.

Comments !