Benchmarking Performance: CPU System Tests

Our first set of tests is our general system tests. These set of tests are meant to emulate more about what people usually do on a system, like opening large files or processing small stacks of data. This is a bit different to our office testing, which uses more industry standard benchmarks, and a few of the benchmarks here are relatively new and different.

All of our benchmark results can also be found in our benchmark engine, Bench.

FCAT Processing: link

One of the more interesting workloads that has crossed our desks in recent quarters is FCAT - the tool we use to measure stuttering in gaming due to dropped or runt frames. The FCAT process requires enabling a color-based overlay onto a game, recording the gameplay, and then parsing the video file through the analysis software. The software is mostly single-threaded, however because the video is basically in a raw format, the file size is large and requires moving a lot of data around. For our test, we take a 90-second clip of the Rise of the Tomb Raider benchmark running on a GTX 980 Ti at 1440p, which comes in around 21 GB, and measure the time it takes to process through the visual analysis tool.

System: FCAT Processing ROTR 1440p GTX980Ti Data

FCAT likes single threaded performance, whcih shows the high frequency parts with faster memory near the top.

Dolphin Benchmark: link

Many emulators are often bound by single thread CPU performance, and general reports tended to suggest that Haswell provided a significant boost to emulator performance. This benchmark runs a Wii program that ray traces a complex 3D scene inside the Dolphin Wii emulator. Performance on this benchmark is a good proxy of the speed of Dolphin CPU emulation, which is an intensive single core task using most aspects of a CPU. Results are given in minutes, where the Wii itself scores 17.53 minutes.

System: Dolphin 5.0 Render Test

Dolphin is also pure ST frequency driven, however a surprise twist in that our Xeon W-2155 beats the Core i7-8086K in this test, although with a margin of error. 

3D Movement Algorithm Test v2.1: link

This is the latest version of the self-penned 3DPM benchmark. The goal of 3DPM is to simulate semi-optimized scientific algorithms taken directly from my doctorate thesis. Version 2.1 improves over 2.0 by passing the main particle structs by reference rather than by value, and decreasing the amount of double->float->double recasts the compiler was adding in. It affords a ~25% speed-up over v2.0, which means new data.

System: 3D Particle Movement v2.1

3DPM likes fast cache and frequency, and the W-2195 is almost fighting with the Core i9-7980XE here, and is let down slightly by its slow memory. The 1950X is still top dog.

DigiCortex v1.20: link

Despite being a couple of years old, the DigiCortex software is a pet project for the visualization of neuron and synapse activity in the brain. The software comes with a variety of benchmark modes, and we take the small benchmark which runs a 32k neuron/1.8B synapse simulation. The results on the output are given as a fraction of whether the system can simulate in real-time, so anything above a value of one is suitable for real-time work. The benchmark offers a 'no firing synapse' mode, which in essence detects DRAM and bus speed, however we take the firing mode which adds CPU work with every firing.

System: DigiCortex 1.20 (32k Neuron, 1.8B Synapse)

DigiCortex is a memory focused benchmark, but can also take advantage of AVX2 and sometimes AVX512, hence why the W-2195 is sat at the top. That being said, it is above the i9-7980XE, despite the latter having dual AVX512 ports.

Agisoft Photoscan 1.3.3: link

Photoscan stays in our benchmark suite from the previous version, however now we are running on Windows 10 so features such as Speed Shift on the latest processors come into play. The concept of Photoscan is translating many 2D images into a 3D model - so the more detailed the images, and the more you have, the better the model. The algorithm has four stages, some single threaded and some multi-threaded, along with some cache/memory dependency in there as well. For some of the more variable threaded workload, features such as Speed Shift and XFR will be able to take advantage of CPU stalls or downtime, giving sizeable speedups on newer microarchitectures.

System: Agisoft Photoscan 1.3.3 (Large) Total Time

Agisoft is a mixture of workloads, although the big multithreaded bit in the middle tends to dominate. Both the W-2195 and W-2155 score the same time, with a cluster of results around it. The Core i9-7960X sits on top though, with a seemingly better mix of cores and threads.

Benchmarking Performance: CPU Office Tests Benchmarking Performance: CPU Rendering Tests
Comments Locked

74 Comments

View All Comments

  • Ian Cutress - Monday, July 30, 2018 - link

    If we ever get a Xeon Scalable system up and running for proper benchmarking, I'll run through some tests.
  • JoJ - Saturday, August 4, 2018 - link

    Hi Ian,

    I beg you to consider the futility of approaching HPE, to obtain a Z station and 56 Scalable cores, max RAM etc, may not may be wholly as anticipated.

    I have a very high profile media production job coming, that could warrant a room of fully loaded such machines. Truly high profile. I have been always able to establish nearly immediate rapport with the marketing folk who are not apparently thought about by the UK online tech media, as warmly as I think it fair, but then I deal in printed worlds of very controlled readerships, and possibly by reflex alone I'm treated like I have more to say than I often do. But approaching the twenty five years mark, I probably ran out of lines some nineteen years ago anyway...

    I care about hurling around heavily laden InDesign files, often without any media placeholders to include 409MB Hassleblad captures straight from the photographer. This project is going to be filled with custom adverts not run in any other media ever. (It took longer to agree copy for Rolls Royce than to sell the deal, in our first encounter with this project. VWAG owned both marques including Bentley, then.) Advertisers may visit, our job to produce for them their first display copy ever, because of interest from services who are in the word of mouth referral ecosystem this publication represents. We are effectively building a advertising studio, to accommodate the needs presented by, eg a long established security company, beyond public profile not by necessity or plan but rather not seeing the circle squared to deliver satisfactory result, without establishing unwanted capacity or staffing. So founder directors shall descend, for interview and portrait, the deal being approved by them signing a proof of the pages, in person, ideally we are able to complete 60% of the interview and layout remotely in advance.

    I want to suggest to you that HPE could genuinely be intrigued by the client base of our project, and want to be exposed to them... mostly boutique banks and funds. A typical user is running a dozen Bloomberg Professional windows, Excel, with live updates and a lot of libraries for giving cell results, will have a terabyte or so of core textual references, between academic papers, email threads, online discussion (usually scraped) , and the job is to fill very specialist magazines with content and advertisers who don't play in the spray and pray brand game, but care everything about how they're found by a prospective new customer. So atop the trappings of a financial trader's desktop, will be layered Word, sure, but a TeX compositor, Visual studio, InDesign, PS...WHY? Because if we can get the message right, we can trade quickly to fill a matching campaign of the same redundancy as big ad buyers get, at even sharper prices. They come for creative, stay for flow execution.

    It's realistic, even commonplace, for a Xeon Scalable workstation to find itself in front of board member powers, in this way.

    Looking at the decisive commitment required to purchase current generation high end workstations, I assume that HPE, Dell et.al. just lean on their direct sales forces.

    but strip away our Corporate veneer, and we're indistinguishable from any of your readers.

    Individual buyers of workstation class computers, commonly inhabit the front line trenches of high end media production. London seethes with freelancer artists, driven by insufficient investment by employers, to fly solo simply to stay current with the tools and turnaround of their art. I can't win any argument about how far from the public imagination of the game, is the typical MB Pro user, sat elegantly in a WeWork chair, I thought my first sight of the Barbican WeWork office, was a film set... (whereas we're actually building a film set where to conduct our FOH business, because I can't vouch for the appearance of our actual digs. (Mr CIO left of his own accord, incapable of tolerating our disinterested views of fancy workspaces... the fact that our dev team is 80% founder level partner, actually never came into play, but the non attendance at his"corporate development" meetings, was protest against the idea of competing for hires with luxuries. Pity, in this one respect alone, the one character aspect we didn't believe necessary to investigate in interview, so patent is our third world architectural infrastructure, relations truly soured only because the guy was so good in every way otherwise. We just couldn't please with such capital commitments, not looking at the impending A grade glut just completing in the City. Such dangerous words; "absolutely, in the future, if you take us there, we will want a statement head office for sales etc". Just not on any immediate plans..

    sorry, Ian, I am avoiding saying that I think I certainly could sell a HPE on the facts we know very well, where they'll be getting sales, in this sector at least, more from physical exposure to buyers and influencers, directly demonstrated the new generation capability by entirely non conveniently situated independent professionals and outside service companies, but absolutely nothing to look at in the kind of performance review I can't really understand is so homogeneous online, because this sort of evaluation hardly relates to the real use at all.

    I am adamant that the entire presentation of high end desktop bixes is hostile and increasingly damaging to sales of performance workstations to corporate office buyers of all kinds.

    Wall to wall gamer imagery has diluted the public perception of utilitarian power of computers beyond homeopathic insignificance.

    Nothing, not a single word for twenty years- it seems to me - has conveyed any part of this crucial message: "With this new Intel computer, and the latest developments in the most advanced quantitative tools for generations, I WHOOP MY COMPETITIONS REAR!'

    By all means, running a renderer matters in the industry that we inhabit. But the reason why we'll shell out for a 56 core beast, is so we can quickly previs a scene, while still working through the latest circulation data dump for the 200,000 or so publications we trade advertising on. Most days, we're rendering visuals of the data we don't want to take our concentration aeay from, again the reason why we're looking at the new HPE Z station Scalable workstations.

    i have to be distracted but I'll conclude with guaranteed brevity upon my short return. There's plenty of angles here to get a vendor to cough up a couple of fully loaded machines for you, and I am entirely serious I'll invest the necessary time to get you a shot if you think I'm not batshii crazy. bfn
  • Elstar - Monday, July 30, 2018 - link

    To be fair, the top end Core X and Xeon W have different TDP values.
  • HStewart - Monday, July 30, 2018 - link

    So does TDP values make bigger impact than actual Frequency - on same chip of course.
  • BurntMyBacon - Tuesday, July 31, 2018 - link

    It is more that TDP and frequency can no longer be separated. If you completely ignore TDP and fix the frequencies to their maximum, then TDP only matters in as much as you have the proper cooling solution to manage it. However, the actual running frequency of Intel's processors are tied into processor temperature, power, number of active cores, etc. On the same chip, a larger TDP means the processor can spend more time at higher frequencies or have more cores active at the same frequency.
  • mode_13h - Monday, July 30, 2018 - link

    The difference relative to X-series is probably due to thermal throttling due to dual AVX-512 and heat buildup under the non-soldered IHS.
  • HStewart - Monday, July 30, 2018 - link

    That would also be interesting test - I believe AVX-512 can be turned off in bios - not sure since I don't have one.
  • JlHADJOE - Tuesday, July 31, 2018 - link

    Lack of Turbo Boost 3.0 maybe?
  • stanleyipkiss - Monday, July 30, 2018 - link

    Besides the VM situation on Threadripper, why would anyone spend $2500+ for this?
  • duploxxx - Monday, July 30, 2018 - link

    what VM situation?

Log in

Don't have an account? Sign up now