AnandTech DAS Suite - Benchmarking for Performance Consistency

Our testing methodology for DAS units takes into consideration the usual use-case for such devices. The most common usage scenario is transfer of large amounts of photos and videos to and from the unit. Other usage scenarios include the use of the DAS as a download or install location for games and importing files directly off the DAS into a multimedia editing program such as Adobe Photoshop. Some users may even opt to boot an OS off an external storage device.

The AnandTech DAS Suite tackles the first use-case. The evaluation involves processing three different workloads:

  • Photos: 15.6 GB collection of 4320 photos (RAW as well as JPEGs) in 61 sub-folders
  • Videos: 16.1 GB collection of 244 videos (MP4 as well as MOVs) in 6 sub-folders
  • BR: 10.7 GB Blu-ray folder structure of the IDT Benchmark Blu-ray

Each workload's data set is first placed in a 25GB RAM drive, and a robocopy command is issued to transfer it to the DAS under test (formatted in NTFS). Upon completion of the transfer (write test), the contents from the DAS are read back into the RAM drive (read test). This process is repeated three times for each workload. Read and write speeds, as well as the time taken to complete each pass are recorded. Bandwidth for each data set is computed as the average of all three passes.

Blu-ray Folder Read

The write workloads see the Extreme PRO v2 come out slightly better than the WD_BLACK P50 using the Haswell testbed. On the reads, we see the Hades Canyon / eGFX enclosure turning out to be better - this can be attributed in part to the capabilities of the testbed itself, rather than the PCIe tunneling chain. In any case, we don't see any significant gulf in the numbers between the different units as long as the observations are made within the USB SuperSpeed 10Gbps or USB SuperSpeed 20Gbps host configurations.We also instrumented our evaluation scheme for determining performance consistency.

Performance Consistency

Aspects influencing the performance consistency include SLC caching and thermal throttling / firmware caps on access rates to avoid overheating. This is important for power users, as the last thing that they want to see when copying over 100s of GB of data is the transfer rate going down to USB 2.0 speeds.

In addition to tracking the instantaneous read and write speeds of the DAS when processing the AnandTech DAS Suite, the temperature of the drive was also recorded at the beginning and end of the processing. In earlier reviews, we used to track the temperature all through. However, we have observed that SMART read-outs for the temperature in NVMe SSDs using bridge chips end up negatively affecting the actual transfer rates. To avoid this problem, we have restricted ourselves to recording the temperature at either end of the actual workloads set. The graphs below present the recorded data.

Performance Consistency and Thermal Characteristics

The first three sets of writes and reads correspond to the photos suite. A small gap (for the transfer of the video suite from the internal SSD to the RAM drive) is followed by three sets for the video suite. Another small RAM-drive transfer gap is followed by three sets for the Blu-ray folder. An important point to note here is that each of the first three blue and green areas correspond to 15.6 GB of writes and reads respectively. There is no issue with thermal throttling - even in the fastest configuration, both the P50 and Extreme PRO v2 show an increase of less than 5C after the workload processing. The P50 seems to have slightly better thermal performance for this workload set.

Synthetic Benchmarks - ATTO and CrystalDiskMark PCMark 10 Storage Bench - Real-World Access Traces
Comments Locked

81 Comments

View All Comments

  • henkhilti - Monday, October 5, 2020 - link

    Why do boot, office, PCMark10 runs.
    DAS is only about copying data from internal to external move the DAS to other system and do the opposite (copy data from external to internal).
    Users don't have ram drives or use robocopy (you already have the synthetic benchmarks).

    Just use fast internal drives (that people actually buy) and start dragging files/folders with File Explorer like humans do :)
  • repoman27 - Monday, October 5, 2020 - link

    I boot from USB external media all the time. It's a pretty common part of many workflows. So is running VM's that reside on external media. Or working with large media files that live on external media dedicated to a particular project / client.
  • hubick - Tuesday, October 6, 2020 - link

    I was bike commuting from home to work with my SSD (Samsung X5 Thunderbolt 3) in my pocket and booting Linux off it at either end. Easier than trying to sync data, plus I only had to maintain the, like, 1000 development tools I need in once place.
  • drajitshnew - Monday, October 5, 2020 - link

    I have been reading Anandtech for 16 years now and this is the most outright confusing article that I have read for the reasons listed
    1. I disagree with the testbed choice-- IT IS NOT A SYSTEM,IT IS A HACK . I do not think that it has realworld applicability. Could you try a couple of Ryzen system. Failing that just stop giving 16 threads q32 results. I cannot think of a case where a 1 TB, bus powered DAS would be used in a realworld use requiring 16*32 random reads. Can you?
    2. I agree with @danneely @spunjji @stormyparis, USB 3.2 is a mess. Even rtfm fails. I would request your reviewers to add the supported speeds to each system, board and flagship mobile review. With 8k video (and limited storage) it is not unthinkable to use one of these devices to transfer files.
  • ganeshts - Monday, October 5, 2020 - link

    The article is a description of our attempts to make use of USB 3.2 Gen 2x2 as a regular consumer. Rarely do people go out on a limb and make a new PC for something that could be achieved with an expansion card. That is the reason for the piece being described as 'ecosystem review', rather than a review of just the WD_BLACK P50 and the SanDisk Extreme PRO v2.

    The testbed choice was an attempt to use the three 'direct-attached storage testbeds' we have used since starting the reviews set back in 2014. It did end up as a 'hack', but that serves the purpose of this particular review well. If you see many more Gen 2x2 reviews using the same Haswell testbed, then your complaint is justified.

    AnandTech's editors do not operate out of a central location. Most (including me) are freelancers spread out throughout the world. As far as testing out a couple of Ryzen systems goes - all the Ryzen PCs with me are SFF machines without a PCIe expansion slot. I am loath to building a new testbed at *this point in time* because USB4 is just around the corner. It is better to build a testbed that can serve the purpose well for at least 2 to 3 years.

    As for bus-powered DAS and the 4KQ32T16 workload - if you don't think it is suitable, feel free to disregard the numbers. The limited applicability of the workload is exactly the reason we felt it was OK to present the results from the other workloads on the same machine.
  • supdawgwtfd - Monday, October 5, 2020 - link

    What editors?
  • drajitshnew - Tuesday, October 6, 2020 - link

    Thank you for addressing my concerns. In light of this I have 2 fresh ones
    1. As you also agree the host system is a hack-- it should not become the regular testbed. I do not bother with T16q32 results in any review, because I do not have any workload that can generate such a workload.
    2. Please again I would like like that all devices with USB 3xyz are labelled with speeds and power output supported.
  • PaulHoule - Monday, October 5, 2020 - link

    The issues they run into here make me think of the practical problems I have with USB.

    I have a few recent Windows laptops that have USB 3 Type A and Type C ports.

    In the old days you were supposed to be able to plug a hub into a host then plug a hub into a hub and do it again and have it work. The spec said you could do it and you really could.

    In the USB3 spec I don't see anything promised as to what kind of configurations are supported and I find I can't take it for granted that I plug my PC into a 4-way hub and can then plug my monitor into the hub and then plug my keyboard and mouse into the monitor.

    Some configurations work but then I plug in the RealSense camera and my mouse stops working; or maybe the SD card reader connects and disconnects all the time and I am always hearing the notification tone for that.

    The hard way I learned "at most four way hubs" and "never plug a hub into a hub" and I've finally settled in on something that works but plug in a USB hard drive and I pray that the filesystem doesn't get corrupted.

    Let's see an expose on that!
  • eastcoast_pete - Monday, October 5, 2020 - link

    I know this will read old-fashioned, but I really wish that one of these USB flavors would allow true "serial" use, i.e. the ability to daisy-chain USB devices directly in the plug. With most "ultraportables" now down to 1-2 ports, USB charging is often made absurd by the then-lost connectivity. If one could simply plug another device into the back of the male USB plug, that issue would be moot. Is there such a solution? I will gladly stay on 3.1 or 3.2 if that feature would be enabled.
  • repoman27 - Monday, October 5, 2020 - link

    In theory, they all do. You just need to embed a USB hub to create that topology. It costs money, adds complexity, and consumes power, but plenty of dongles and some chargers already do this.

Log in

Don't have an account? Sign up now