Multi-Client CIFS Performance for Professional Workloads

NAS units used in SMBs / SMEs need to provide good performance under heavy load from multiple clients. The SPEC SFS 2014 benchmark uses real-life workloads (just like Intel NASPT), but makes it easier for users to understand the benchmark results. This is achieved by using the concept of business metrics. Given a particular NAS unit, how many concurrently accessed databases can reside in it? How many IP cameras or video streams can be simultaneously recorded? To determine this metric, each load point is associated with a target required op rate. If the NAS under test doesn't meet that op rate, it is deemed as an 'invalid run'. SPEC requires all published benchmarks to follow certain strict rules - such as presented results having no invalid runs for at least 10 load points. Unfortunately, small-scale NAS systems with 7200 RPM drives can't meet these requirements, Hence, we can't officially publish SPEC SFS 2014 benchmark results for the evaluation of the Asustor AS6204T.

Using a popular filer benchmarking program, we did play back multi-client real-world professional workload access traces on the NAS using up to 10 Windows 7 VMs. The Asustor AS6204T with four 7200 RPM hard drives in RAID-5 can support,at any given point of time, recording of 10 or more video streams, or, up to five databases, or, up to four software builds, or, up to three virtual desktops.

Database Operations

The Database Operations workload seems to get / doesn't seem to get acceptable performance for up to 5 clients.The detailed metrics from our trace playback are available here

Database Operations - Op Rates

Database Operations - Bandwidth and Latencies

Software Builds

The Software Builds workload seems to get / doesn't seem to get acceptable performance for up to 4 clients.The detailed metrics from our trace playback are available here

Software Builds - Op Rates

Software Builds - Bandwidth and Latencies

Video Recording

The Video Recording workload seems to get / doesn't seem to get acceptable performance for more than 10 clients.The detailed metrics from our trace playback are available here

Video Recording - Op Rates

Video Recording - Bandwidth and Latencies

Virtual Desktops

The Virtual Desktops workload seems to get / doesn't seem to get acceptable performance for up to 3 clients.The detailed metrics from our trace playback are available here

Virtual Desktops - Op Rates

Virtual Desktops - Bandwidth and Latencies

Multi-Client CIFS Performance for Consumer Workloads Multi-Client NFS Performance for Professional Workloads
Comments Locked

29 Comments

View All Comments

  • RdVi - Thursday, November 5, 2015 - link

    It's hard to compare the graphs on page 4 when the values on the axis vary (sometimes wildly) between the two. Flicking back and forth it is very hard to easily compare on face value. Would it not be possible to have the scale locked to whichever graph approaches the highest value?
  • ganeshts - Thursday, November 5, 2015 - link

    The issue here is that these graphs are generated on a per-review basis - i.e, the graph that you see for the QNAP TS-451+ was generated at the time of writing the QNAP TS-451+ review. At that time, the Asustor AS6204T hadn't been evaluated, so it is not possible to know ahead of time what the highest value is going to be.

    I think we have the best possible solution here in the timeframe that can be devoted to the review - the other solution is to simply not have any graphs for the competing units (similar to what we did for the QNAP TS-451+ review which was the first 4-bay one done with the new methodology).
  • DanNeely - Friday, November 6, 2015 - link

    Are the graphs coming from the bench software itself? If there is a CSV export option generating combined comparison graphs (or rescaled stand alone ones) shouldn't be hard to do after the fact.
  • ganeshts - Friday, November 6, 2015 - link

    The bench software is written by me :) (except for the core trace replay component which is compiled from NASPT source). And, yes, the graphs are generated at the end of each benchmark run by my code.

    I don't have a CSV export option in the software yet, as I only export the HTML tables that are linked above each graph. So, yes, the data is there, but not in a format suitable for reprocessing with each new NAS run.

    It is possible to implement CSV export and graph regeneration, but it is a matter of how much time I can spend to get the new code up and running. At this moment, I think the time cost far outweighs the benefits provided. I am currently working on optimizing our SPEC SFS testbed and benchmarking - I think that will provide more benefits to readers in the long run.
  • Oyster - Friday, November 6, 2015 - link

    Ganesh, I mentioned this in some of your other NAS reviews. You need to review the software side of these vendors now. Would be awesome to see the biggies go head to head... Synology, QNAP, ASUS, and Lenovo. Please try to cover app offerings to other items like backdooring via SSH, etc.

    As usual, keep up the good work.
  • ganeshts - Friday, November 6, 2015 - link

    Yes, it is in the works. The problem is that these are 'subjective' aspects and require more effort to use and record observations compared to objective benchmarking.
  • MrElectrifyer - Saturday, November 7, 2015 - link

    Curious how this compares to a QNAP TS-470 Pro...
  • MrElectrifyer - Saturday, November 7, 2015 - link

    So far, I know the TS-470 Pro can support more than 8GB of RAM...I upgraded mine to 10GB.
  • WatcherCK - Monday, November 9, 2015 - link

    Looks like disks get formatted internally to Ext4... The full specifications link seems to point back to the articles initial page was this by design? The full spec pdf for this NAS can be found here: http://www.asustor.com/service/resource?type=3&... (for a laugh check out what the display on the NAS is saying on its Asustor page, what do you know Asus? :) )

    Anyone considering a DIY NAS should checkout the new Asrock Rack C236 WSI that combined with a Silverstone DS380 would make for a powerful little storage box/virtualization server... They dont seem to be available anywhere yet unfortunately and Supermicro only have MATX C236 motherboards.

Log in

Don't have an account? Sign up now