A Look At Triple-GPU Performance And Multi-GPU Scaling, Part 1
by Ryan Smith on April 3, 2011 7:00 AM ESTIt’s been quite a while since we’ve looked at triple-GPU CrossFire and SLI performance – or for that matter looking at GPU scaling in-depth. While NVIDIA in particular likes to promote multi-GPU configurations as a price-practical upgrade path, such configurations are still almost always the domain of the high-end gamer. At $700 we have the recently launched GeForce GTX 590 and Radeon HD 6990, dual-GPU cards whose existence is hedged on how well games will scale across multiple GPUs. Beyond that we move into the truly exotic: triple-GPU configurations using three single-GPU cards, and quad-GPU configurations using a pair of the aforementioned dual-GPU cards. If you have the money, NVIDIA and AMD will gladly sell you upwards of $1500 in video cards to maximize your gaming performance.
These days multi-GPU scaling is a given – at least to some extent. Below the price of a single high-end card our recommendation is always going to be to get a bigger card before you get more cards, as multi-GPU scaling is rarely perfect and with equally cutting-edge games there’s often a lag between a game’s release and when a driver profile is released to enable multi-GPU scaling. Once we’re looking at the Radeon HD 6900 series or GF110-based GeForce GTX 500 series though, going faster is no longer an option, and thus we have to look at going wider.
Today we’re going to be looking at the state of GPU scaling for dual-GPU and triple-GPU configurations. While we accept that multi-GPU scaling will rarely (if ever) hit 100%, just how much performance are you getting out of that 2nd or 3rd GPU versus how much money you’ve put into it? That’s the question we’re going to try to answer today.
From the perspective of a GPU review, we find ourselves in an interesting situation in the high-end market right now. AMD and NVIDIA just finished their major pushes for this high-end generation, but the CPU market is not in sync. In January Intel launched their next-generation Sandy Bridge architecture, but unlike the past launches of Nehalem and Conroe, the high-end market has been initially passed over. For $330 we can get a Core i7 2600K and crank it up to 4GHz or more, but what we get to pair it with is lacking.
Sandy Bridge only supports a single PCIe x16 link coming from the CPU – an awesome CPU is being held back by a limited amount of off-chip connectivity; DMI and a single PCIe x16 link. For two GPUs we can split that out to x8 and x8 which shouldn’t be too bad. But what about three GPUs? With PCIe bridges we can mitigate the issue some by allowing the GPUs to talk to each other at x16 speeds and dynamically allocate CPU-to-GPU bandwidth based on need, but at the end of the day we’re splitting a single x16 lane across three GPUs.
The alternative is to take a step back and work with Nehalem and the x58 chipset. Here we have 32 PCIe lanes to work with, doubling the amount of CPU-to-GPU bandwidth, but the tradeoff is the CPU. Gulftown and Nehalm are capable chips on its own, but per-clock the Nehalem architecture is normally slower than Sandy Bridge, and neither chip can clock quite as high on average. Gulftown does offer more cores – 6 versus 4 – but very few games are held back by the number of cores. Instead the ideal configuration is to maximize performance of a few cores.
Later this year Sandy Bridge E will correct this by offering a Sandy Bridge platform with more memory channels, more PCIe lanes, and more cores; the best of both worlds. Until then it comes down to choosing from one of two platforms: a faster CPU or more PCIe bandwidth. For dual-GPU configurations this should be an easy choice, but for triple-GPU configurations it’s not quite as clear cut. For now we’re going to be looking at the latter by testing on our trusty Nehalem + x58 testbed, which largely eliminates a bandwidth bottleneck in a tradeoff for a CPU bottleneck.
Moving on, today we’ll be looking at multi-GPU performance under dual-GPU and triple-GPU configurations; quad-GPU will have to wait. Normally we only have two reference-style cards of any product on hand, so we’d like to thank Zotac and PowerColor for providing a reference-style GTX 580 and Radeon HD 6970 respectively.
97 Comments
View All Comments
Ryan Smith - Sunday, April 3, 2011 - link
I am not highly sensitive to microstuttering (aliasing on the other hand...). In my experience nothing here microstuttered, and the only thing that performed poorly was the NV 3xGTX580 setup under Bad Company 2.james.jwb - Sunday, April 3, 2011 - link
Ryan, do you think you could re-run just a single game (you choose) but with the CPU overclocked to 4.0Ghz or higher if you can (4.5ghz if possible). 3 GPU's will surely react well to this, I'd love to see the results (for 2x, too).james.jwb - Sunday, April 3, 2011 - link
just at 2560 btw :)DanNeely - Sunday, April 3, 2011 - link
Ryan posted elsewhere that the 920 he's using is from the slow end of the bell curve and isn't stable above 3.33; so until he gets a new system this is as good as it gets.PhantomKnight - Sunday, April 3, 2011 - link
This is not the only way. There are flexible PCIe cables around on eBay. I don't know how well they would work, but it would be possible to use something like this to increase the room around the cards. http://cgi.ebay.com.au/PCI-E-PCI-Express-16X-Riser... ETC7Enigma - Tuesday, April 5, 2011 - link
Interesting. I wonder if this would harm latency somehow (or reduce the amount of power the PCIe slot can supply due to essentially using an extension cord for the gpu).masterkritiker - Sunday, April 3, 2011 - link
While it is a great article, sadly it's a limited one. The dual/triple GPU test should have been done across multiple platforms, x58/890FX/P67 regardless of some platform limited pcie lanes like the P67. Let's be honest SNB systems are the ones selling like mad right now, you'll hardly convince more people w/ an x58 system to spend money upgrading their GPU to dual/triple GPUs while SNB-E is just around the corner & their x58 systems are still great for them. Trust me people will "wait"! The test could have efficiently validated anyone deciding to upgrade from single to dual/triple GPUs across multiple platforms in either a single 1080p/1600p monitor or even multiple monitors, I guess we have to wait for the test(If that will happen).b1u3 - Monday, April 4, 2011 - link
2x590GTX vs 2x6990...ypsylon - Monday, April 4, 2011 - link
Frankly I can't get my head around this test. If you have ~2000$ to blow on 3 VGAs then there is good chance you will buy 2x590* or 2x6990. It is more logical and convenient choice.Performance by a tiny fraction lower vs 3xSLI/CF (at worst - only with reference - under-clocked - models currently available), but you'll need only 2 cards, not 3. Easier to implement, less clutter inside, and if by any chance you own a motherboard with 3* slot space between 1st and 2nd x16 slot then it is absolute win-win.
piroroadkill - Monday, April 4, 2011 - link
Quite. I don't know why 6990 CF, just like 5970 CF, was almost completely ignored by AnandTech..