CPU Tests: Legacy and Web

In order to gather data to compare with older benchmarks, we are still keeping a number of tests under our ‘legacy’ section. This includes all the former major versions of CineBench (R15, R11.5, R10) as well as x264 HD 3.0 and the first very naïve version of 3DPM v2.1. We won’t be transferring the data over from the old testing into Bench, otherwise it would be populated with 200 CPUs with only one data point, so it will fill up as we test more CPUs like the others.

The other section here is our web tests.

Web Tests: Kraken, Octane, and Speedometer

Benchmarking using web tools is always a bit difficult. Browsers change almost daily, and the way the web is used changes even quicker. While there is some scope for advanced computational based benchmarks, most users care about responsiveness, which requires a strong back-end to work quickly to provide on the front-end. The benchmarks we chose for our web tests are essentially industry standards – at least once upon a time.

It should be noted that for each test, the browser is closed and re-opened a new with a fresh cache. We use a fixed Chromium version for our tests with the update capabilities removed to ensure consistency.

Mozilla Kraken 1.1

Kraken is a 2010 benchmark from Mozilla and does a series of JavaScript tests. These tests are a little more involved than previous tests, looking at artificial intelligence, audio manipulation, image manipulation, json parsing, and cryptographic functions. The benchmark starts with an initial download of data for the audio and imaging, and then runs through 10 times giving a timed result.

We loop through the 10-run test four times (so that’s a total of 40 runs), and average the four end-results. The result is given as time to complete the test, and we’re reaching a slow asymptotic limit with regards the highest IPC processors.

(7-1) Kraken 1.1 Web Test

Google Octane 2.0

Our second test is also JavaScript based, but uses a lot more variation of newer JS techniques, such as object-oriented programming, kernel simulation, object creation/destruction, garbage collection, array manipulations, compiler latency and code execution.

Octane was developed after the discontinuation of other tests, with the goal of being more web-like than previous tests. It has been a popular benchmark, making it an obvious target for optimizations in the JavaScript engines. Ultimately it was retired in early 2017 due to this, although it is still widely used as a tool to determine general CPU performance in a number of web tasks.

(7-2) Google Octane 2.0 Web Test

Speedometer 2: JavaScript Frameworks

Our newest web test is Speedometer 2, which is a test over a series of JavaScript frameworks to do three simple things: built a list, enable each item in the list, and remove the list. All the frameworks implement the same visual cues, but obviously apply them from different coding angles.

Our test goes through the list of frameworks, and produces a final score indicative of ‘rpm’, one of the benchmarks internal metrics.

We repeat over the benchmark for a dozen loops, taking the average of the last five.

(7-3) Speedometer 2.0 Web Test

Legacy Tests

(6-5a) x264 HD 3.0 Pass 1(6-5b) x264 HD 3.0 Pass 2

(6-4a) 3DPM v1 ST(6-4b) 3DPM v1 MT

(6-3a) CineBench R15 ST(6-3b) CineBench R15 MT

CPU Tests: Simulation CPU Tests: Synthetic
Comments Locked

98 Comments

View All Comments

  • Threska - Friday, July 16, 2021 - link

    One could have more than one.

    https://www.reddit.com/r/hardware/comments/mc8j2c/...

    Especially depending upon the memory type.

    https://semiengineering.com/what-designers-need-to...
  • croc - Monday, July 19, 2021 - link

    The first year anniversary of the Threadripper Pro! And a timely review to celebrate it! Not the CPU, the release... Because that might be the last Threadripper we see for quite some time....

    I get the impression that AMD has been in the position of runer up for so long now that they don't know how to capitalize on a lead. Either that or by selling off their fab they have lost touch with how to design for fabrication, which might explain their failure to deliver product at the 7NM node. Unfilled Epycs, no Zen 3 Threadrippers...
  • mode_13h - Monday, July 19, 2021 - link

    What a croc!
    ; )

    > that might be the last Threadripper we see for quite some time....

    The announcement for Chagall is rumored to be coming in August, with volume shipping in September. Have you heard otherwise?

    > by selling off their fab they have lost touch with how to design for fabrication

    Huh? Why do you think the issue is design-related?

    And if we're contemplating counter-factuals, then let's not lose sight of the fact that GF is still on 12 nm. And if AMD still owned them, we should also consider whether the whole enterprise would still be doing business, at all.

    > which might explain their failure to deliver product at the 7NM node.

    Or maybe they're in the same boat as everyone else, facing higher demand and restricted supply?
  • croc - Tuesday, July 20, 2021 - link

    Lemmee see.... Wasn't Chagall an artist? And wasn't the code name for the Threadripper Zen 3 to be Genesis Peak? And aren't you quoting a rumor? Have there been ANY leaked benchmarks? Has ANY processor EVER launched without leaked benchmarks?

    The lack of things often speak volumes...
  • mode_13h - Tuesday, July 20, 2021 - link

    > And aren't you quoting a rumor?

    Yes, that's exactly what I said.

    > Have there been ANY leaked benchmarks?
    > Has ANY processor EVER launched without leaked benchmarks?

    It's a niche product. There typically aren't many motherboard options for them (and even fewer, at launch). I think it's not surprising, if there aren't any benchmark leaks as of yet. There won't be many engineering samples floating around.

    Anyway, we'll know within about 5 weeks if there's any truth to the rumor.
  • croc - Wednesday, July 21, 2021 - link

    I will know when a bios with AGESA support for a Zen 3 Threadripper arrives. None have arrived yet, and typically they do so about a month prior. Chagal is an artist, and artist names are usually reserved for apus. Supposedly the next TR will use the TRX40 socket, which means that there are many MBs for testing / leaking. Given that the Epyc still uses the SP3 socket, I believe this to be tue The latest 'chagall' rumor is that it won't release before Sept, mebbe as late as November. Gotta love them movable goal posts

    Personaly, I don't think that AMD can get the cores to deliver at the frequencies required for an HEDT product Hell, given their back orders for their server chip I doubt that they even care. Bigger fish to fry, all that. Still, a bit of egg-on-face for Dr. Su. And niche? don't tell the CGI world that they are niche...
  • Qasar - Wednesday, July 21, 2021 - link

    sorry croc, but you could be wrong, looks like TR based on zen 3 could be Chagall, and on both STRX4 and SWRX8, at least according to here :
    https://www.guru3d.com/news-story/amd-threadripper...
    https://www.notebookcheck.net/Zen-3-based-Ryzen-Th...

    from moores law is dead, a video about TR zen 3 :
    https://www.youtube.com/watch?v=la-7Q_VsWUM
    but no concrete info as of yet, but a search on google for threadripper zen 3, all seems to say the same, code name chagall
  • croc - Saturday, July 24, 2021 - link

    No new BIOS since April... 7 days to August, so looks like another no-show, eh...
  • Qasar - Saturday, July 24, 2021 - link

    and the point is ? considering their has been no official announcement from amd when zen 3 TR is to be released, why would there be a bios for it yet ?
    the point of my reply was that you could be wrong on the code name, not the release date.

    at least amd HAS a HEDT cpu, when was the last one from intel ?

    bottom line : all there are, are rumors, which should be taken with some salt.
  • croc - Monday, July 26, 2021 - link

    That would be the Xeon w-3175x

Log in

Don't have an account? Sign up now