Server Buying Decisions: Memory
by Johan De Gelas on December 19, 2013 10:00 AM EST- Posted in
- Enterprise
- Memory
- IT Computing
- Cloud Computing
- server
Worth the Price Premium?
The real question is thus whether LRDIMMs are worth the 60% higher cost per GB. Servers that host CPU-bottlenecked (HPC) applications are much better off with RDIMMs, as the budget should be spent on faster CPUs and servers as much as possible. The higher speed that LRDIMMs offer in certain configurations may help for some memory intensive HPC applications, but the memory buffer of LRDIMMs might negate the clock speed advantage as it introduces extra latency. We will investigate this further in the article, but it seems that most HPC applications are not the prime target for LRDIMMs.
Virtualized servers are the most obvious scenario where the high capacity of LRDIMMs can pay off. As the Xeon E5 V2 ("Ivy Bridge EP") increased the core count from maximum 8 to 12, many virtualized servers will run out of memory capacity before they can use all those cores. It might be wiser to buy half as many servers with twice as much memory. A quick price comparison illustrates this:
- An HP DL380 G8 with 24 x 16GB RDIMMs, two E5-2680v2, two SATA disks and a 10 GbE NIC costs around $13000
- An HP DL380 G8 with 24 x 32GB LRDIMMs, two E5-2680v2, two SATA disks and a 10 GbE NIC costs around $26000
At first sight, buying twice as many servers with half as much memory is more attractive than buying half as many servers with twice as much capacity. You get more processing power and more network bandwidth and so on. But those advantages are not always significant in a virtualized environment.
Most software licenses will make you pay more as the server count goes up. The energy bill of two servers with half as much memory is always higher than one server with twice as much memory. And last but not least, if you double the amount of servers, you will increase the time you spend on administering the server cluster.
So if your current CPU load is relatively high, chances are that an LRDIMM equipped server makes sense: the TCO will be quite a bit lower. We have tested this in our previous article and found that having more memory available can reduce the response time of virtualized applications significantly even if you're running at high CPU load. Since that test, little has changed, besides the fact that LRDIMMs have become a lot cheaper. So it is pretty clear that for virtualized clusters, LRDIMMs have become a lot more attractive.
Besides a virtualized cluster, there is another prime candidate: servers that host disk limited workloads, where memory caching can alleviate the bottleneck. Processing power is irrelevant in that case, as the workload is dominated by memory and/or disk accesses. Our Content Delivery Network (CDN) server test is a real world example of this and will quantify the impact of larger memory capacity.
Facebook
Twitter
RSS
27 Comments
View All Comments
subflava - Thursday, December 19, 2013 - link
Great article...look forward to more enterprise/IT professional based articles from Anandtech in the future. This is very timely for me as my company is just about to pull the trigger on a server upgrade. Interesting stuff.JohanAnandtech - Friday, December 20, 2013 - link
Thanks for sharing! :-)DERSS - Friday, December 27, 2013 - link
You guys are seriously super-cool; thanks.wsaenotsock - Thursday, December 19, 2013 - link
costed?blaktron - Thursday, December 19, 2013 - link
Good article, although as an enterprise architect, I can tell you the one true benefit to LRDIMMS is in 2 and 4 socket vhost builds, because the double density RAM gives you the freedom to turn off NUMA spanning and still get near-ideal guest density.Almost nobody runs caching servers that big, although at almost double performance over a 256GB build (the 100k + concurrent user norm) its kind of attractive to run 2 of these per DC instead of 6 smaller ones (which would actually be the real world comparison with those kind of deltas).
mexell - Thursday, December 19, 2013 - link
Real-world pricing, at least in the enterprise context, is quite a bit off from your numbers. In my employer's price bracket, we regularily buy similar servers as your 24*16GB config for about the same price (13k€) - but including a 3 year subscription VMWare Enterprise license, which is about 6 to 7 k€ on its own. No one pays list price on that kind of hardware.JohanAnandtech - Friday, December 20, 2013 - link
Are you sure that there is not a big discount on the VMware license? And smaller enterprises will pay something close to the list price. I know that the typical discount is 10-20% for smaller quantities, not more.blaktron - Friday, December 20, 2013 - link
Depends on the country Johan. The partner channel managers get to decide discounts on partner orders (which he is describing). Also, the bundling discount doesn't happen everywhere, but I could buy that server for like $15k CDN.The VMware license cost seems out of this world to me too, because we license our hosts for anyone from 2500 to 5k CDN, depending on their agreement with VMware.
mexell - Saturday, December 21, 2013 - link
I don't really know where exactly the discount is applied, as the licenses are OEM and we don't get line-item pricing. In our market segment (large enterprise with Dell, medium-to-large with HP) we usually see at least 40% off on list prices, in some cases (networking equipment) up to 75%.VMWare, on the other hand, is especially rigid with their pricing structure. Two years ago, when we negotiated for a 100 host branch office deployment, they referred to their list pricing. For them, we are not even big enough to speak directly to us.
dstarr3 - Thursday, December 19, 2013 - link
Wow. With 768GB of memory, I bet you could run Crysis.