Meltdown & Spectre: Analyzing Performance Impacts on Intel's NUC7i7BNH
by Ganesh T S on March 23, 2018 4:15 PM EST- Posted in
- Systems
- Spectre
- Benchmarks
- Meltdown
One of the consequences of the Meltdown and Spectre vulnerabilities, besides the immediate security concerns, is what it has meant for our ongoing testing and benchmarking operations. Along with accounting for the immediate performance changes from the required patches, we've also needed to look at how we present data and how we can (or can't) compare new patched systems to older unpatched systems. From an editorial perspective we need to ensure the continuity of our data as well as providing meaningful benchmark collections.
What we've found so far is that the impact of the Meltdown and Spectre patches varies with the workload and the type of test. Nate's GPU testing shrugs it off almost entirely, while Billy's SSD testing especially feels the pinch. In the middle of course are consumer system reviews, where workloads are more varied but also much more realistic and often more relevant as well. Of course, this is also the type hardware that we most often have to return when we're done, making it the most challenging to correct.
As everyone at AnandTech has been integrating the patches and updating their datasets in their own way, I wanted to offer some insight into what's going on for system testing. With several important systems set to launch in the first half of this year – particularly Intel's Hades Canyon – I've been going back and collecting updated results for use in those future reviews. In the process, I also wanted to document how performance has been impacted by these security patches and which benchmarks in particular have been affected.
Evaluation Setup
The Intel Kaby Lake NUC (NUC7i7BNH) was used for all the benchmarks presented in this article. It was chosen for two reasons. Being a relatively new system that consumers can still purchase in the market, evaluating it presents data of particular relevance to readers. Intel was prompt in providing BIOS updates to resolve the Meltdown and Spectre issues compared to other OEMs whose systems I had in hand for evaluation. The NUC was configured with the following specifications:
| Intel NUC7i7BNH (Unpatched) Specifications | |
| Processor | Intel Core i7-7567U Kaby Lake-U, 2C/4T, 3.5 - 4.0 GHz, 14nm+, 4 MB L2, 28W TDP |
| Memory | Crucial Ballistix Sport LT BLS16G4S240FSD.16FAD DDR4 15-15-15-35 @ 2133 MHz 2x16 GB |
| Graphics | Intel Iris Plus Graphics 650 |
| Disk Drive(s) | ADATA SX8000NP (128 GB; M.2 Type 2280 PCIe 3.0 x4 NVMe; Micron 32L 3D MLC) |
| Networking | Intel Dual Band Wireless-AC 8260 (2x2 802.11ac - 866 Mbps) 1x Intel 10/100/1000 RJ-45 GbE |
| Audio | 3.5mm Headphone Jack Capable of 5.1/7.1 digital output with HD audio bitstreaming (HDMI) |
| Miscellaneous I/O Ports | 4x USB 3.0 Type-A 1x Thunderbolt 3 USB-C 1x micro-SDXC |
| Operating System | Retail unit is barebones, but we installed Windows 10 Enterprise x64 |
| Pricing (Barebones) | USD 445 |
| Full Specifications | Intel NUC7i7BNH Specifications |
Three different patch configurations were benchmarked:
- Completely unpatched
- Only OS patches activated (Graph entry : Patch OS)
- Completely patched with updates to both the OS and the CPU microcode (Graph entry : Patch OS + uCode)
While it will (eventually) be an uncommon configuration, right now a lot of systems are still only running OS patches without their associated microcode updates, as the latter is still being distributed. At the same time, because the microcode patches are primarily for Spectre – Meltdown fixes are implemented at an OS level – this allows us to see just how much of an impact the Spectre fixes have on top of the existing Meltdown fixes.
The completely unpatched system used Windows 10 version 16299.125 with BIOS v0054. As evident from the screenshot below, the system is susceptible to both Spectre and Meltdown.
The second configuration was processed with Windows 10 version 16299.309 with BIOS v0060. The screenshot below shows that the system is protected against Meltdown, but, not Spectre.
The final fully patched configuration was process with Windows 10 version 16299.214 and BIOS v0061. The current BIOS is v0062 (the one we used was available briefly before it was pulled to undergo additional QA for possible random restarts). This configuration shows full protection against both Meltdown and Spectre.
We used our standard test suite for low power desktops / industrial PCs in order to evaluate the impact of the patching on the performance of the system.
Facebook
Twitter
RSS
83 Comments
View All Comments
iter - Monday, March 26, 2018 - link
According to whom? You, the workstation all-seer? Or perhaps some statistics done over the internet?iter - Monday, March 26, 2018 - link
Also, if a "standalone system" is for you the opposite of "connected to the internet" that is quite indicative... You know there exists this thing called a network, on top of which the internet runs. You can have a load of workstations and servers in a network that is not connected to the outside world.Most places that do important work do it this way. Eliminates 99.99% of threats from the outside and the from the inside. Just one of many other common sense things, such as disabled usb storage devices, unauthorized network clients and whatnot. Machines that do connect to the internet are physically isolated from the secure network. They use secure proprietary interfaces for explicit data transfer between the two networks under tight scrutiny.
rhoades-brown - Tuesday, March 27, 2018 - link
Eh? So, your saying that you would put your workstations unpatched and completely unprotected on a network where other devices can connect to it?Did you hear about WannaCrypt? Your network connected workstation would have been easy prey.
Would you allow these unprotected workstations to share files with other workstations and what about the cheaper machines? I assume that you are either creating or processing content/data of some description. Have a look at MS16-120 - 'The most serious of these vulnerabilities could allow remote code execution if a user either visits a specially crafted website or opens a specially crafted document.'
What about USB sticks? Something on one machine could easily be spread to another, and people are stupid enough to plug in a USB stick that they found in a car park, etc.
There are exceptions- air-gaped networks to make things highly secure, but that seems unlikely, and if your workstations are in that rare scenario, have a look at xLED which uses a compromised switch to flash it's status LEDs to share data- crazy, I know; scary, absolutely.
Gasaraki88 - Monday, March 26, 2018 - link
Wow, that's a big exaggeration...Bulat Ziganshin - Friday, March 23, 2018 - link
>Though there is a certain irony to the fact that taken to its logical conclusion, patching a CPU instead renders storage performance slower, with the most impacted systems having the fastest storage.It looks ironic because it was incorrectly attributed as CPU bug. But the point is that it allows to discover information when OS allows it, and thus it's an OS bug of not preventing it. As far as you run pure CPU computations, it doesn't need any mitigations.
The only thing that need to be patched is communication between OS and application, and therefore you got larger hit when these communications are more intensive - on higher-IOPS operations. So f.e. I/O in large blocks (1 MB or so) is unaffected, but 4K I/O is affected, especially with higher-performance drives and higher QD scenarios.
jordanclock - Friday, March 23, 2018 - link
It is a CPU bug. The speculative execution is faulty and that is a CPU feature. The OS patches are simply workarounds to prevent certain kinds of speculative execution.Reflex - Friday, March 23, 2018 - link
It is not a bug at all at either level. It is a feature that was found to be able to be abused. That happens all the time. Once found, it was mitigated, in this case by disabling the feature (Meltdown) or mitigating the impact (Spectre). In future designs it will be mitigated or eliminated.There are all sorts of features your CPU is capable of utilizing that can compromise your data or stability (hey, you can still run in unprotected mode for memory!), when it is found to be a problem it is typically disabled at the appropriate level (microcode/firmware/OS).
bji - Friday, March 23, 2018 - link
Uh, no. It's a feature that comes with an unintended side effect of allowing data reads that should be disallowed. That part of it is a bug, plain and simple. I guess you are the kind of person that would call a bug that crashes the computer a "feature" because "it saves you power when your PC is off because it crashed".PixyMisa - Friday, March 23, 2018 - link
So it's a bug.yeeeeman - Saturday, March 24, 2018 - link
Bug is something that doesn't work as designed. I am pretty sure that they designed and verified it this way. These vulnerabilities are not bugs, they are just security loopholes.