Virtualization Performance: Linux VMs on ESXi

We introduced our new vApus FOS (For Open Source) server workloads in our review of the Facebook "Open Compute" servers. In a nutshell, it is a mix of four VMs with open source workloads: two PhpBB websites (Apache2, MySQL), one OLAP MySQL "Community server 5.1.37" database, and one VM with VMware's open source groupware Zimbra 7.1.0. Zimbra is quite a complex application as it contains the following components:

  • Jetty, the web application server
  • Postfix, an open source mail transfer agent
  • OpenLDAP software, user authentication
  • MySQL is the database
  • Lucene full-featured text and search engine
  • ClamAV, an anti-virus scanner
  • SpamAssassin, a mail filter
  • James/Sieve filtering (mail)

All VMs are based on a minimal CentOS 6 setup with VMware Tools installed. All our current virtualization testing is on top of the hypervisor which we know best: ESXi (5.0). We have changed two things in our vApusMark FOS setup: we upgradeded the guestOS from 5.6 to 6.0 and increased the number of vCPUs of the OLAP VM from 2 to 4. This small upgrade means that our latest results should not be compared to the results in our older articles. We test with four tiles (one tile = four VMs). Each tile needs nine vCPUs, so the test requires 36 vCPUs.

vApusMark FOS

For being just a minor update, the new Piledriver core does pretty well. Clock for clock performance goes up by 11%. The total performance gain (IPC+clock) is 20%, which is significant. The Opteron 6376 performs only 4% better than its direct competitor the E5-2630 (as the latter will perform very similar to our E5-2660 with 6 cores), but that is not bad at all: you get slightly better performance for a lower (server) price.

The top of the line 6380 cannot keep up with the best Xeons. Offering 86% of the more expensive Xeon E5-2660 is hardly a disaster, however. Note "maximum amount of affordable memory" is on top of many virtualization hosts shopping lists followed by price/performance. For those buyers, considering that a server based upon the Opteron cost less, the Opteron is once again a potent virtualization host if the power usage is similar.

With the lack of c-states, the Opteron 6174 did pretty poorly. The Opteron 6276 consumed a lot less at idle than its predecessor, but consumed a lot more when pressured to perform at high load. So we were very keen to learn whether AMD has improved power consumption too. Did AMD finally get that part right?

Benchmarking Configuration Measuring Real-World Power Consumption
Comments Locked

55 Comments

View All Comments

  • sherlockwing - Wednesday, February 20, 2013 - link

    These Piledriver based Opterons look competitive but the threat of Ivy-EP is immenient. The last time Intel die-shrunk their High end platform they introduced the monsterous 10 core Westmere-EP(the current Xeon E7 lineup), I wouldn't be surprised Ivy-EP introduces 10/12 core extreme E7 Xeons as well as Octa Xeons with better performance/watt.
  • Kevin G - Wednesday, February 20, 2013 - link

    Ivy Bridge-E is indeed coming but it is looking to be 6 months out. These Opterons were shipping since November which would give them a 10 month lead time. The real question for AMD is what they'll have in response in that time frame. Steamroller based parts all look to be released in 2014. On the bright side, AMD should be pairing those chips with a new socket as DDR4 becomes available.

    One thing though about Ivy Bridge-E is that it will also be a socket 2011 part so migration to it should get relatively quick in comparison to the Westmere-EP to Sandybridge-E transition. The same cost savings for OEM noted in this article for socket G32 Opterons will apply to Ivy Bridge-E this time around.
  • Oskars Apša - Wednesday, February 20, 2013 - link

    Wasn't intels 2011 socket to be only physically identical, but electrically totally redesigned?
  • Hrel - Friday, February 22, 2013 - link

    "These Opterons were shipping since November"

    I reject this statement. Nothing counts as being "on the market" until Anandtech has done a full review of it. That's my stance and I'm sticking to it :P
  • Beenthere - Wednesday, February 20, 2013 - link

    ...is that the 63xx series is focused primarily on micro servers where it fits well. If the just disclosed Jaguar cores are any indication of AMD products to be released this and next year, I'd say AMD is back in the game in many PC and portable markets.

    The only thing Ivy Bridge has going for it is reduced power but at a price penalty.
  • JohanAnandtech - Thursday, February 21, 2013 - link

    SeaMicro was indeed one of first to use Piledriver based cores, but I don't think the Opteron 6300 is meant to be a "typical" microserver CPU. Otherwise, AMD would have focused more on low power parts. This meant to be an update for the general server market.
  • Jovec - Wednesday, February 20, 2013 - link

    ... as it is showing the multi-threaded chart instead.
  • JohanAnandtech - Wednesday, February 20, 2013 - link

    Fixed. Thanks for pointing it out, always appreciated.
  • Death666Angel - Wednesday, February 20, 2013 - link

    Hey!
    I get a " Page Not Found" error from the Racktivity PDU link. :)
  • ssj3gohan - Wednesday, February 20, 2013 - link

    You say that AMDs bad implementation of C6 costs them in the energy efficiency tests, but AFAIK with a low of still 10% CPU the CPU should not enter ACPI C3 (Intel C6), it will probably stay in C1e providing there is still more than enough workload to do on each OS tick.

    If the xeons are observed to go into ACPI C3, then that is very probably a scheduler optimization specific for intel processors, not an actual implementation problem by AMD. Balancing C-state transitions - especially complete core sleep modes like ACPI C3 - is a notoriously hard task to do because each transition also costs a certain amount of mJ that, on immediate wake, are wasted compared to just leaving the cores in C1(e)

Log in

Don't have an account? Sign up now