BAPCo Publishes SYSmark 2018

One of the bigger consortiums for developing benchmarks is BAPCo. Over the years the company has released a variety of benchmarking tools, such as MobileMark, TabletMark, and SYSmark. This latter software tool is run by a number of high profile OEMs to categorize their systems, and used by numerous government agencies to gauge performance on systems with key metrics. Today BAPCo is announcing its next generation of SYSmark, known as SYSmark 2018.

The BAPCo consortium is a non-profit consortium, with members including Dell, HP, Hitachi, Intel, Lenovo, Microsoft, Pegatron, Samsung, Toshiba, WD, Winstron, and others. The goal of the consortium is to develop objective performance benchmarks, using high profile and common tools as the baseline for the metrics. By virtue of being a high-profile non-profit, and working at government level procurement, BAPCo has had significant success in its benchmark deployments.

Sysmark 2018 Interface, Ryzen 7 2700X Result

Software such as SYSmark 2018 takes a number of software utilities, such as Photoshop, Office, Chrome, and PowerDirector to profile a system with four metrics: an overall score, a productivity score, a creativity score, and a responsiveness score. Recent updates in the last edition enabled power measurement during benchmarks using a Watts Up! meter, which also comes through in the 2018 edition. The new edition has the following tests (taken from the manual):

1. Productivity

The Productivity scenario models productivity usage including word processing (mail merge, document comparison, and PDF conversion), spreadsheet data manipulation (data modeling, financial forecasting), and email creation/management, presentation editing, software development (compiling code), application installation, and archiving files.

• Adobe® Acrobat® Pro DC
• AutoIT 3.3.14.2
• BowPad64-2.3.3 installer
• Google Chrome™
• Microsoft® Excel® 2016
• Microsoft® OneNote® 2016
• Microsoft® Outlook® 2016
• Microsoft® PowerPoint® 2016
• Microsoft® Word® 2016
• Windows Zip

2. Creativity

The Creativity scenario models editing digital photos (creating HDR and panoramic photos), cataloguing digital photos (organizing catalog, use of facial detection to group people), and editing digital video (create a timeline from various source clips and transcode the output).

• Adobe® Photoshop® CC 2017
• Adobe® Lightroom® Classic CC
• CyberLink PowerDirector® 2015

3. Responsiveness

The Responsiveness scenario includes activities such as multiple application launches, web browsing with many tabs open, and task switching.

• Adobe® Acrobat® Pro DC
• Adobe® Lightroom® Classic CC
• Adobe® Photoshop® CC 2017
• CyberLink PowerDirector® 2015
• Google Chrome™
• Microsoft® Excel® 2016
• Microsoft® OneNote® 2016
• Microsoft® Outlook® 2016
• Microsoft® PowerPoint® 2016
• Microsoft® Word® 2016

Standard System Comparison

These benchmarks blur the lines between synthetic and real-world, though often land on the side of real-world. By condensing the results into a single number, it can offer an easy way to reflect on the scale of performance, although it also obfuscates some of the nuance.  The results are compared to a base system score of 1000 points, which for this generation will be a low-end Dell Optiplex 5050 Tower:

• Processor: Intel® Core™ i3-7100 @ 3.90 GHz
• Operating System: Microsoft Windows 10 Pro x64 (build 1709)
• RAM: 1x4GB DDR3
• Graphics: Intel® HD Graphics 630
• Audio: Integrated High Definition Audio
• Storage: 128GB capacity, M.2 SATA
• Networking: Integrated Gigabit Ethernet

This is a step up from SYSmark 2014 1.5, which used an i3-6100 and a spinning hard-drive. With this default system, the i3-7100 has a nice and fast single core performance, is a dua core with hyperthreading, but will be let down by the single channel memory.

New features for SYSmark 2018 include the ability to uninstall the software, run the tests from the command line, and also enable post-processing of sub-test data. All three of which should help reviewers and testers automate into our scripts.

We ran the benchmark on a number of test systems we have set up, and scored the following:

BAPCo’s benchmark suites are approved by the consortium members before being released. It must be pointed out that AMD left the consortium after SYSmark 2012 was launched (NVIDIA and VIA also left). This came about due to disagreements about the nature and weighting of the workloads that seemed beneficial to the competition. Despite this, SYSmark is still considered by many as a major benchmark for various markets, and since AMD’s newest high performance x86 processor came to market, results in the last generation SYSmark 2014 SE were competitive.

Related Reading

• Basemark Releases New "Basemark GPU" Mobile and PC Benchmark
• Futuremark To Release VRMark Cyan Room on November 22: A DX12 VR Benchmark
• A First Look at Futuremark’s New 3DMark Time Spy Extreme: DX12 Benchmark Now in 4K
• VRScore: Crytek and Basemark Announce New Benchmark Release