LRDIMMs, RDIMMs, and Supermicro's Latest Twin
by Johan De Gelas on August 3, 2012 4:45 AM EST- Posted in
- IT Computing
- Intel
- Samsung
- Xeon
- Cloud Computing
- Supermicro
HyperCloud DIMMs
There is a fourth option: Netlist's HCDIMM. Netlist, a company that specializes in making VLP (Very Low Profile) memory, offers an alternative for LRDIMMs: HyperCloud DIMMs.
Instead of using a centralized buffer (LRDIMMs), HCDIMMs use a distributed buffer to reduce the electrical load on the memory channels. Combined with an HCDIMM register, four ranks are presented as two.
HCDIMM advantages
The advantage of HCDIMMs is that HCDIMMs run one speed bump faster than LRDIMMs. So while LRDIMMs have to throttle back to 1066MHz at 3 DPC, HCDIMMs run at 1333MHz. According to a Netlist sponsored report, HCDIMMs offer about 17% higher bandwidth, which sounds reasonable to us. Secondly, the distributed buffer architecture is the same architecture that DDR4 is converging on. However DDR4 will be running at a much lower 1.2V.
HCDIMM Disadvantages
The combination of a register and distributed buffers (and clock redundancy) comes with a serious drawback: power. Although we could not test HCDIMMs ourselves, most industry sources talk about 20% higher power than LRDIMMs, while LRDIMMs only consume more than RDIMMs with 1 DPC. In 3 DPC configurations, LRDIMMS consume about the same as RDIMMs.
Secondly, HCDIMMs are not a JEDEC standard. As a result the HCDIMM ecosystem—module, server and CPU vendors—is smaller. AMD and Intel do not officially validate HCDIMMs, so server vendors have do the complete validation effort themselves. IBM for example only offers them in one system (IBM system x3650 M4), and 16GB HCDIMMs are quite a bit more expensive than both 16GB LRDIMMs and RDIMMs. LRDIMMs are much more widespread; almost every server vendors supports them in a wide range of server models.
Lastly, HCDIMMs are only available from one module vendor, Netlist.
Nevertheless, HCDIMMs are viable but somewhat expensive alternative in the high capacity memory market if your server supports them. Netlist has been quite successful in convincing the server vendors: several models of HP, Supermicro, and Gigabyte support HCDIMMs. HP offers HCDIMMs in their most popular servers (DL380/DL360), although HCDIMMs can only be installed by HP.
Once Netlist gets the 32GB parts out in large volumes, the extra competition can probably drive prices down. Until then, HCDIMMs offer only a speed advantage over RDIMMs.
Overview
Let us structure all this info in a table.
| DIMM Types, Speed, and Capacity Limitations | ||||||
| DIMM type | UDIMM | RDIMM | RDIMM LV | LRDIMM | HCDIMM | |
| Maximum speed at 2 DPC | 1333MHz | 1600MHz | 1333MHz | 1333MHz | 1600MHz | |
| Maximum speed at 3 DPC | Not possible | 1333MHz | 1333MHz | 1066MHz | 1333MHz | |
| Maximum capacity per CPU (Quad channel) | 64GB |
192GB (3 DPC) 256GB (2 DPC)* |
192GB (3 DPC) 256GB (2 DPC)* |
384GB | 192GB | |
| Top speed at maximum capacity | 1066 |
1066 (DR) ** 800 (QR) ** |
1066 (DR) ** 800 (QR) ** |
1066 | 1333 | |
| Voltage | 1.5V | 1.5V | 1.35 V | 1.35V/1.5V | 1.35V/1.5V | |
| loaded, 3 DPC Power usage (at 1.5V) per DIMM | 4 W | 4.5 W | <= 4 W | 5-6 W | 8-9 W | |
| Intel CPU support |
Xeon 5500 Xeon 5600 Xeon E5 |
Xeon 5500 Xeon 5600 Xeon E5 |
Xeon 5600 Xeon E5 Xeon E7 |
Xeon 5600 Xeon E5 Xeon E7 |
N/A | |
| AMD CPU support |
Opteron 4000 Opteron 6100 Opteron 6200 |
Opteron 4000 Opteron 6100 Opteron 6200 |
Opteron 6100 Opteron 6200 |
Opteron 6200 | N/A | |
| Server support | all servers | all servers | all servers | Dell, HP, IBM,Supermicro |
HP DL360G8 DL380G8 IBM x3650 M4 Supermicro |
|
* Quad Rank DIMMs require special BIOS support and validation and are not available on all servers.
** DR = Dual Rank, QR = Quad Rank.
1600MHz LRDIMMs are possible but not commercially available as far as we know. 32GB HCDIMMs are available in very small quantities. LRDIMMs are available from almost every server module manufacturer out there, while HCDIMMs are netlist modules only.
Facebook
Twitter
RSS
26 Comments
View All Comments
koinkoin - Friday, August 3, 2012 - link
For HPC solutions I like the Dell C6220, dense, and with 2 or 4GB of memory per cpu core you get a good configuration in a 2U chassis for 4 servers.But for VMware, servers like the R720 give you more room to play with memory and IO slots.
Not counting that those dense server don’t offer the same level of management and user friendliness.
JohanAnandtech - Friday, August 3, 2012 - link
A few thoughts:1. Do you still need lots of I/O slots now that we can consolidate a lot of gigabit Ethernets in Two 10GBe
2. Management: ok, a typical blade server can offer a bit more, but the typical remote management solutions that Supermicro now offers are not bad at all. We have been using them for several years now.
Can you elaborate what you expect from the management solution that you won't expect to see in a dense server?
alpha754293 - Friday, August 3, 2012 - link
re: network consolidationNetwork consolidation comes at a cost premium. You can still argue that an IB QDR will give you better performance/bandwith, but a switch is $6k and other systems that don't have IB QDR built in, it's about $1k per NIC. Cables are at least $100 a piece.
If you can use it and justify the cost, sure. But GbE is cheap. REALLY REALLY cheap now that it's been in the consumer space for quite some time.
And there aren't too many cases when you might exceed GbE (even the Ansys guys suggest investing in better hardware rather than expensive interconnects). And that says a LOT.
re: management
I've never tried Supermicro's IMPI, but it looks to be pretty decent. Even if that doesn't work, you can also use 3rd party like logmein and that works quite well too! (Although not available for Linux, but there are Linux/UNIX options available out there as well).
Supermicro also has an even higher density version of this server (4x half-width, 1U DP blade node.)
JonBendtsen - Monday, August 6, 2012 - link
I have tried Supermicro IPMI, works nicely. I can power on/off the machine and let it boot from a .iso image I have on my laptop. This means that in case I have to boot from a rescue CD, then I do not even have to plug a CD drive into the machine. Everything can be done from my laptop, even when I am not in the office, or even the country.bobbozzo - Tuesday, August 7, 2012 - link
Can you access boot screens and the BIOS from the IPMI?For Linux, I use SSH (or VNC server), but when you've got memory or disk errors, etc., it's nice to see the BIOS screens.
Bob
phoenix_rizzen - Thursday, August 9, 2012 - link
Using either the web interface on the IPMI chip itself, or the IPMIView software from SuperMicro, you get full keyboard, mouse, console redirection. Meaning, you can view the POST, BIOS, pre-boot, boot, and console of the system.You can also configure the system to use a serial console, and configure the installed OS to use a serial console, and then connect to the serial console remotely using the ipmitool program.
The IPMI implementation in SuperMicro motherboards (at least the H8DG6/H8DGi series, which we use) is very nice. And stable. And useful. :)
ForeverAlone - Friday, August 3, 2012 - link
Only 128GB RAM? Unacceptable!Guspaz - Monday, August 20, 2012 - link
It starts to matter more when you're pouring on the VMs. With two sockets there, you're talking 16 cores, or 32 threads. That's the kind of machine that can handle a rather large number of VMs, and with only 128GB of RAM, that would be the limitation regarding how many VMs you could stick on there. For example, if you wanted to have a dedicated thread per VM, you're down to only 4GB per VM, which is kind of low for a server.darking - Friday, August 3, 2012 - link
I think the price on the webpage is wrong. or atleast it differs by market.i just checked the Danish and the British webstores, and the 32GB LRDIMMS are priced at around 2200$ not the 3800$ that the US webpage has.
JohanAnandtech - Friday, August 3, 2012 - link
They probably changed it in the last few days as HP as lowered their price to $2000 a while ago. But when I checked, it was $3800