The Xserve Server Platform

The most surprising and even astonishing results of the previous article were, of course, the MySQL and Apache server benchmarks. A powerful Windows XP based client (see above: "Client Configuration: Dual Opteron 250") fires off an enormous amount of Select, grouping and ordering read intensive queries and simulates 1 to 50 concurrent clients. All that query data is sent over a direct Gigabit Ethernet link to the tested server; in this case, a PowerMac Dual G5 2.5 GHz running OS X Server (Tiger). In part I, we discovered that performance of the Apple machine completely collapsed once there were more than 2 concurrent clients.

The solution? Install a Linux distribution to verify our suspicion that the OS is to blame is on the mark. We chose Yellow Dog Linux (YDL). Terra Soft, the company behind Yellow Dog, is an Apple Authorized OEM Value Added Reseller, so you could say that Apple has no objection to installing YDL on your Apple machines. There is more: Terra Soft is specialized in optimizing for the G5 processor. The version that we used, Yellow Dog Linux 4.0.1, is based on the Linux Kernel version 2.6.10-1.ydl.1g5-smp.

Let us see how the Dual 2.5 GHz G5 performed in MySQL when running Yellow Dog Linux. Please note: YDL 4.0 wouldn't run on the 2.7 GHz Apple machine, so we do not have results for that platform.

The difference between the PowerMac running Linux and Mac OS X Server is absolutely striking. Mac OS X server shows better performance going from one to a second connection (and thus thread) because the second CPU steps in and helps carry the load. After that, however, performance completely collapses and stabilizes at around 50 queries per second.

While the G5 is not the best integer processing unit out there, it is not the one to blame for the poor performance that we experienced in our first tests. Running Yellow Dog Linux, the Dual G5 was capable of performing similar to a 3 GHz Xeon. Notice that more concurrent connections gives better performance from 1 to 20. At 5 concurrent simulated users, YDL simply wipes the floor with Mac OS X: 411 versus 113 queries per second. It gets worse at 10 concurrent users: 443 queries per second on Linux versus 62 on Mac Os X. Around 20 connections, performance declines only very slowly just like all the x86/Linux machines.

With the MySQL performance woes now clearly caused by OS X, let us see if Apache tells us the same story. We tested with Apachebench, with "n" being the total of number of connections and "c" the total of concurrent connections:
ab -n 100000 -c 100 http://localhost
Some people suggested that we should test with both Apache 1.3 and 2.0, so we gave Apache 2.0 a test run.

Unit: Requests per second Powermac Dual G5 2.5 GHz OS X Powermac Dual G5 2.5 GHz YDL Dual Xeon 3.6 GHz
Apache 1.3 250 709 1291
Apache 2.0 266 2165 3410

On OS X, we noticed that the activity monitor was telling us that the CPUs were not working very hard and were underutilized. This seems to indicate that the problem with Apache is somewhat different from MySQL, as MySQL showed a CPU load between 165% and 190%. (200% is the maximum, as there are 2 CPUs in the system.)

Apple told us that the problem lies in Apachebench (the client side), which stalls from time to time and thus generates too low of a load on the (Apache) server. The weird thing is that this does not happen with few connections (up to 10,000). When we repeated the test, Apachebench on Mac OS X gets in trouble again. Version 2.0 is slightly faster on OS X, but it still trails by a significant margin. On the other hand, YDL and the Xeon platform are roughly 3X as fast with version 2.0.

According to Apple, this is a bug in Apachebench. Now, we can accept that explanation, as it is clear that the server is not loaded and can still accept a lot more web requests. However, the Apachebench problem is still interesting. Why exactly does the client stall? Is it really a bug or is it running out of some resources? We didn't delve deeper, as we are developing a less synthetic, closer to the real world benchmark to test web servers.

Even if we ignore the Apache results, our MySQL tests - and the queries used in these tests - are based on a real world usage pattern of a real world database. The G5 is partially crippled by a chipset that takes a long time to access the memory, and it's not the fastest integer CPU; still, it performs like a 3 GHz Xeon on Linux. The problem clearly lies in Mac OS X, and is worth further investigation.

Micro CPU Benchmarks: Isolating the FPU Bottleneck Search
Comments Locked


View All Comments

  • stmok - Thursday, September 1, 2005 - link

    LOL...As everyday passes, it seems more "interesting things" are revealed from Apple solutions.
  • ViRGE - Thursday, September 1, 2005 - link

    Granted, some of this was over my head(more than I'd like to admit to), but your results are none the less very interesting Johan. Now that we have the Linux/G5 numbers, there's no arguing that there's a weakness in MacOSX somewhere, which is a bit depressing as a Mac user, but still a very useful insight as to how there's obviously something very broken in some design aspect of the OS(it simply shouldn't be getting crushed like it is). My only question now is how Apple and its devs will respond to this - it is pretty damning after all.

    Thanks for finally getting some Linux/G5 numbers out to settle this.
  • sdf - Friday, September 2, 2005 - link

    By changing hardware platforms.

    No, seriously.

    A transition from PowerPC to Intel would be the perfect time to correct ABI flaws like this. It isn't that the G5 causes the slow down, it's that the slow down (maybe) can't really be fixed without breaking binary compatibility. A CPU transition is clearly going to do that anyway, so maybe they'll just wait...
  • toelovell - Thursday, September 1, 2005 - link

    I am kind of curious to see how Darwin would work on an x86 based system for these same tests. There are x86 binaries for Darwin 8. So it should be possible to run these tests and compare Darwin with Linux on an x86 platform. This would help to see if the OS really is the limitation. Just a thought.
  • JohanAnandtech - Thursday, September 1, 2005 - link

    If linux is capable of pushing the G5 8 times higer than with Mac OS X, there is little doubt on my mind that the OS is the problem. Or did I understand you wrong?

    Anyway, I have no experience whatsoever with Darwin. My first impression is that installing Darwin on x86 is probably a very masochistic experience, due to lack of proper drivers. We might get it working but can it really run MySQL and other apps? THere are probably libraries missing... Will the results be representative of anything as it is probably tuned for just getting it running instead of performance? Anyone with Darwin x86 experience?
  • wjcott - Thursday, September 1, 2005 - link

    The only interest I have in a mac OS is if they are going to sell it without a computer. I would love to have OS X, but I must build the machine.
  • Quanticles - Thursday, September 1, 2005 - link

    Every component must be fine tuned to the upmost degree... Every BIOS Setting... Every Hidden Register... *crazy eyes* =)

Log in

Don't have an account? Sign up now