AMD Launches NVMe RAID Support For Ryzen Threadripper Platformby Billy Tallis on October 2, 2017 9:00 AM EST
A week later than originally planned, AMD is launching NVMe RAID support for their Ryzen Threadripper platform. Grab the Windows 10 drivers and release notes here. This updated RAID capability should work with any X399/Threadripper motherboard and any NVMe SSDs, enabling RAID modes 0, 1 and 10 for up to 10 NVMe SSD connected to the CPU's PCIe lanes, but a motherboard firmware update may be required first.
AMD's Ryzen Threadripper launched with support for RAID modes 0, 1 and 10 for arrays of SATA drives connected through the X399 chipset's eight SATA ports. The X399 chipset doesn't offer RAID capability for NVMe SSDs, but given its limited PCIe lane counts, this isn't much of an inconvenience. The Ryzen Threadripper processors themselves have plenty of PCIe lanes to go around. In a typical configuration, the motherboard will have up to three M.2 slots providing PCIe x4 connections to the Threadripper CPU (leaving 48 PCIe lanes for GPUs and four lanes for the connection to the X399 chipset). These PCIe x4 ports can also operate as SATA ports to support M.2 SATA SSDs. However, since these ports are provided by the CPU and not the chipset, they're not covered by the RAID capabilities of the chipset.
To allow for RAID arrays of NVMe SSDs, the Ryzen Threadripper platform needed another RAID implementation. Nobody has manufactured a true NVMe hardware RAID controller—one that completely abstracts the individual drives in the array and presents a single NVMe device to the host. Like all previous NVMe RAID implementations and almost all motherboard RAID implementations for SATA drives, AMD's NVMe RAID for Ryzen Threadripper is at heart a software RAID system. That means it needs new drivers for the OS, and matching support in the motherboard's UEFI to enable booting from a RAID array.
Chipset RAID implementations for SATA typically rely on switching the SATA controller out of AHCI mode and into a proprietary RAID mode. This ensures that the operating system's standard storage drivers won't be able to access the drives, so there's no danger of the array being accidentally corrupted by an OS that isn't aware the drive is part of an array. When Intel added NVMe RAID to their consumer platform with the Sunrise Point 100-series chipsets accompanying Skylake processors, they included a similar mechanism that hides NVMe devices that are connected through the chipset and instead makes them accessible through non-standard interfaces on the SATA controller.
For Intel's X299 HEDT platform for Skylake-X, the same chipset RAID capabilities are available, but the more interesting feature is Intel's Virtual RAID on CPU (VROC) for NVMe SSDs attached to the CPU's PCIe lanes. VROC is a software RAID system that builds atop the Intel Volume Management Device (VMD) feature of Skylake-X and Skylake-SP processors. VMD allows some of the CPU's PCIe ports to be partitioned off into a separate PCI domain behind a virtual PCI bridge. VMD has largely been described by Intel as a feature intended to simplify support for hotplug and enclosure management (status LEDs) of NVMe SSDs, but it apparently also helps build a robust software RAID stack. Intel has not shared much technical information about how VROC works, but they have announced that its features will be segmented by requiring the purchase of a hardware key to unlock RAID capability, and a premium key to unlock RAID-5.
AMD has not described any features of their Zen architecture that sound particularly similar to Intel's VMD or the NVMe remapping feature of Intel's consumer platform chipsets, so it's not clear what hardware features (if any) are helping to enable their NVMe RAID implementation. From a user perspective, setting up NVMe RAID is very similar to setting up chipset SATA RAID: configure the array through the motherboard firmware and load an extra driver when installing Windows. Unlike Intel's plans for VROC, AMD isn't trying for any extra product segmentation and isn't charging extra for any of their NVMe RAID functionality.
One important warning: the new RAID system is incompatible with the previous SATA RAID system for X399. Users with an existing SATA RAID array will not be able to access it using the new RAID drivers. The only migration path AMD currently offers is to re-create the array using the new drivers and firmware, and restore the data from a back-up. For users booting off a RAID array, this means re-installing Windows.
Post Your CommentPlease log in or sign up to comment.
View All Comments
peevee - Monday, October 2, 2017 - linkHow are they going to create RAID10 (which requires at least 4 disks) from "up to 3" NVMe drives?
And without RAID5, 3 SSD RAIDs are not possible at all.
JoeyJoJo123 - Monday, October 2, 2017 - linkAt least Ryzen Threadripper has NVMe RAID enabled, unlike (cough) Intel.
Gothmoth - Monday, October 2, 2017 - linkand i suppose you can use PCIe adapater cards who support 2 to 8 nvme ssd´s.
duploxxx - Tuesday, October 3, 2017 - linkhahah you totally screw the purpose of NVMe going pci-e adaptors ....
BurntMyBacon - Tuesday, October 3, 2017 - linkHow so?
I'd agree that you totally screw the purpose of going M.2 by using PCIe adapters, but some people are only using M.2 because it is the most common NVMe form factor on the market. I've never been a fan of the M.2 form factor for desktops/workstations/servers as both U.2 and PCIe SSDs have better cooling and U.2 doesn't take up so much area on the board. M.2 makes a lot of sense for laptops and thin-mITX, but I don't think we'll see Threadripper on either of those platforms particularly soon. When manufacturers other than just Intel decide to start making U.2 SSDs, you'll be a lot more likely to see support for larger NVMe RAID arrays on motherboards.
barleyguy - Wednesday, October 4, 2017 - linkNo, you really don't.
NVMe SSDs are the same bus bandwidth as PCIE Gen 3 x4. So a 16x PCIE adapter can run 4 of them, if the bios will support 4+4+4+4.
Derbauer has a video on YouTube running 8x 960 Pro NVMe drives on 2x 4 slot cards. The combined bandwidth of 8 960 pros is actually faster than a channel of the memory controller, so pretty insanely fast. (The memory controller on Threadripper is 85 GB per second according to CPU World.)
Xpl1c1t - Thursday, October 5, 2017 - linkfaster throughput, but how about the latency?
ah... one day we will have a non-volatile unified storage technology that replaces NAND and DRAM
IGTrading - Monday, October 2, 2017 - linkIt seems there is a typo in the sentence "The X399 chipset doesn't offer RAID capability for NVMe SSDs, but given its limited PCIe lane counts, this isn't much of an inconvenience. " .
The author probably talks about X370, not X399.
hulu - Monday, October 2, 2017 - linkNo, this news piece is about X399 and Threadripper. The implied comparison here is to Intel's platforms where the chipsets have more lanes, and do offer NVMe RAID.
mdriftmeyer - Monday, October 2, 2017 - link128 PCI-E lanes on Threadripper. Where's the bottleneck?