Server Buying Decisions: Memory
by Johan De Gelas on December 19, 2013 10:00 AM EST- Posted in
- Enterprise
- Memory
- IT Computing
- Cloud Computing
- server
Benchmark method
We used the HP DL 380 Gen8, the best selling server in the world. The good people at Micron helped ensure we could test with 24 DIMMs.
As we were told that the new Xeon E5-26xx V2 ("Ivy Bridge EP") had better support for LRDIMMs than the Xeon E5-26xx ("Sandy Bridge EP"), we tested with both the E5 2680 v2 and the E5-2690 in the Stream and latency tests. For the real-world CDN test (see further) we only use the E5-2680 v2. As the latter test is not limited by the CPU at all (25% load on the E5-2680v2), testing with different CPUs does not make much sense.
Benchmark Configuration: HP DL 380 Gen8 (2U Chassis)
Testing with the HP-DL 380 Gen8 was a very pleasant experience: changing CPUs and DIMMs is extremely easy and does not involve screwdrivers.
The benchmark configuration details can be found below.
| CPU |
Two Intel Xeon processor E5-2680 v2 (2.8GHz, 12c, 25MB L3, 115W) Two Intel Xeon processor E5-2690 (2.9GHz, 8c, 20MB L3, 135W) |
| RAM |
768GB (24x32GB) DDR3-1866 Micron MT72JSZS4G72LZ-1G9 (LRDIMM) or 384GB (24 x 16GB) Micron MT36JSF2G72PZ – DDR3-1866 |
| Internal Disks | One Intel MLC SSD710 200GB |
| BIOS version | P70 09/41/2013 |
| NIC | Dual-port Intel X540-T2 10Gbit NC |
| PSU | HP 750W CS Platinum Plus Hot Plug 750 Watt |
A dual-port Intel X540-T2 10Gbit NC was installed and was connected to the DELL PowerConnect 8024F switch with a 20Gbit bond.
Facebook
Twitter
RSS
27 Comments
View All Comments
JohanAnandtech - Friday, December 20, 2013 - link
First of all, if your workload is read intensive, more RAM will almost always be much faster than any flash cache. Secondly, it greatly depends on your storage vendor whether adding more flash can be done at "dramatically lower cost". The tier-one vendors still charge an arm and a leg for flash cache, while the server vendors are working at much more competitive prices. I would say that in general it is cheaper and more efficient to optimize RAM caching versus optimizing your storage (unless your are write limited).blaktron - Friday, December 20, 2013 - link
Not only are you correct, but significantly so. Enterprise flash storage at decent densities is more costly PER GIG than DDR3. Not only that, but you need the 'cadillac' model SANs to support more than 2 SSDs. Not to mention fabric management is a lot more resource intensive and more prone to error.Right now, the best bet (like always) to get performance is to stuff your servers with memory and distribute your workload. Because its poor network architecture that creates bottlenecks in any environment where you need to stuff more than 256GB of RAM into a single box.
hoboville - Friday, December 20, 2013 - link
Another thing about HPC is that, as long as a processor has: enough RAM to do its dataset on the CPU/GPU before it needs more data, the quantity of RAM is enough. Saving on RAM can let you buy more nodes, which gives you more performance capacity.markhahn - Saturday, January 4, 2014 - link
headline should have been: if you're serving static content, your main goal is to maximize ram per node. not exactly a shocker eh? in the real world, at least the HPC corner of it, 1G/core is pretty common, and 32G/core is absurd. hence, udimms are actually a good choice sometimes.mr map - Monday, January 20, 2014 - link
Very interesting article, Johan!I would very much like to know what specific memory model (brand, model number) you are referring to regarding the 32GB LRDIMM—1866 option.
I have searched at no avail.
Johan? / Anyone?
Thank you in advance!
/ Tomas
Gasaraki88 - Thursday, January 30, 2014 - link
A great article as always.ShirleyBurnell - Tuesday, November 5, 2019 - link
I don't know why people are still going after server hardware. I mean it's the 21st century. Now everything is on cloud. Where you have the ability to scale your server anytime you want to. I mean the hosting provider companies like: AWS, DigitalOcean, Vultr hosting https://www.cloudways.com/en/vultr-hosting.php, etc. has made it very easy to rent your server.