Meltdown & Spectre: Analyzing Performance Impacts on Intel's NUC7i7BNH
by Ganesh T S on March 23, 2018 4:15 PM EST- Posted in
- Systems
- Spectre
- Benchmarks
- Meltdown
One of the consequences of the Meltdown and Spectre vulnerabilities, besides the immediate security concerns, is what it has meant for our ongoing testing and benchmarking operations. Along with accounting for the immediate performance changes from the required patches, we've also needed to look at how we present data and how we can (or can't) compare new patched systems to older unpatched systems. From an editorial perspective we need to ensure the continuity of our data as well as providing meaningful benchmark collections.
What we've found so far is that the impact of the Meltdown and Spectre patches varies with the workload and the type of test. Nate's GPU testing shrugs it off almost entirely, while Billy's SSD testing especially feels the pinch. In the middle of course are consumer system reviews, where workloads are more varied but also much more realistic and often more relevant as well. Of course, this is also the type hardware that we most often have to return when we're done, making it the most challenging to correct.
As everyone at AnandTech has been integrating the patches and updating their datasets in their own way, I wanted to offer some insight into what's going on for system testing. With several important systems set to launch in the first half of this year – particularly Intel's Hades Canyon – I've been going back and collecting updated results for use in those future reviews. In the process, I also wanted to document how performance has been impacted by these security patches and which benchmarks in particular have been affected.
Evaluation Setup
The Intel Kaby Lake NUC (NUC7i7BNH) was used for all the benchmarks presented in this article. It was chosen for two reasons. Being a relatively new system that consumers can still purchase in the market, evaluating it presents data of particular relevance to readers. Intel was prompt in providing BIOS updates to resolve the Meltdown and Spectre issues compared to other OEMs whose systems I had in hand for evaluation. The NUC was configured with the following specifications:
| Intel NUC7i7BNH (Unpatched) Specifications | |
| Processor | Intel Core i7-7567U Kaby Lake-U, 2C/4T, 3.5 - 4.0 GHz, 14nm+, 4 MB L2, 28W TDP |
| Memory | Crucial Ballistix Sport LT BLS16G4S240FSD.16FAD DDR4 15-15-15-35 @ 2133 MHz 2x16 GB |
| Graphics | Intel Iris Plus Graphics 650 |
| Disk Drive(s) | ADATA SX8000NP (128 GB; M.2 Type 2280 PCIe 3.0 x4 NVMe; Micron 32L 3D MLC) |
| Networking | Intel Dual Band Wireless-AC 8260 (2x2 802.11ac - 866 Mbps) 1x Intel 10/100/1000 RJ-45 GbE |
| Audio | 3.5mm Headphone Jack Capable of 5.1/7.1 digital output with HD audio bitstreaming (HDMI) |
| Miscellaneous I/O Ports | 4x USB 3.0 Type-A 1x Thunderbolt 3 USB-C 1x micro-SDXC |
| Operating System | Retail unit is barebones, but we installed Windows 10 Enterprise x64 |
| Pricing (Barebones) | USD 445 |
| Full Specifications | Intel NUC7i7BNH Specifications |
Three different patch configurations were benchmarked:
- Completely unpatched
- Only OS patches activated (Graph entry : Patch OS)
- Completely patched with updates to both the OS and the CPU microcode (Graph entry : Patch OS + uCode)
While it will (eventually) be an uncommon configuration, right now a lot of systems are still only running OS patches without their associated microcode updates, as the latter is still being distributed. At the same time, because the microcode patches are primarily for Spectre – Meltdown fixes are implemented at an OS level – this allows us to see just how much of an impact the Spectre fixes have on top of the existing Meltdown fixes.
The completely unpatched system used Windows 10 version 16299.125 with BIOS v0054. As evident from the screenshot below, the system is susceptible to both Spectre and Meltdown.
The second configuration was processed with Windows 10 version 16299.309 with BIOS v0060. The screenshot below shows that the system is protected against Meltdown, but, not Spectre.
The final fully patched configuration was process with Windows 10 version 16299.214 and BIOS v0061. The current BIOS is v0062 (the one we used was available briefly before it was pulled to undergo additional QA for possible random restarts). This configuration shows full protection against both Meltdown and Spectre.
We used our standard test suite for low power desktops / industrial PCs in order to evaluate the impact of the patching on the performance of the system.
Facebook
Twitter
RSS
83 Comments
View All Comments
Samus - Saturday, March 24, 2018 - link
It isn't a bug, or a design flaw. It's just an exploit of the architecture.Saying otherwise is like saying houses not built for category 3 hurricanes in New York have a design flaw when the area has never needed construction to that spec. But with climate change, the need is becoming necessary as the architecture is no longer fit for the climate.
Not a great analogy, but in the same example, neither Intel nor construction designers anticipated the architecture would become flawed due to unforeseen circumstances.
Alexvrb - Saturday, March 24, 2018 - link
It's definitely a flaw. Unintentional security flaws are still flaws. It's an exploit of a security flaw in the architecture. They'll release a CPU in the future which still have speculative execution and isn't vulnerable to this flaw, at an architecture level. It might have other flaws, however. CPU flaws, bugs, errata are very common.HStewart - Saturday, March 24, 2018 - link
"It isn't a bug, or a design flaw. It's just an exploit of the architecture."It is attempt to distract on Intel productions, that also back fire because it also effects ARM and AMD.
bji - Sunday, March 25, 2018 - link
That's not really true. There is a class of very hard to exploit design flaws in most implementations of speculative execution that seem to be systemic to almost all chips, that is true. These are so hard to exploit as to be nearly unexploitable in my opinion. These are called Spectre.But there is also a much more significant and easy to exploit design flaw. This affects only Intel chips (and I guess some ARM chips too -- but not AMD). This is called Meltdown.
So there is no distracting going on here. Almost all chips are affected by spectre, but it's so hard to exploit as to almost be irrelevant. Meltdown is serious and it's Intel only.
Manch - Monday, March 26, 2018 - link
HStewart is an Intel shill/fan boy. In his mind Intel can do no wrong. You're wasting your breath arguing with him.bcronce - Sunday, March 25, 2018 - link
A bug is when something does not work to spec.boozed - Sunday, March 25, 2018 - link
I bet you $10 the spec doesn't say "our branch prediction should have this massive security flaw".linuxgeex - Monday, March 26, 2018 - link
Correct, it's a design flaw not a bug. When something operates as designed it isn't a bug, but that doesn't make it correct either. Intel is rightly getting sued by the people who are significantly affected and who can afford to battle Intel in court... that will be people who forked over millions to obtain small performance improvements based on Intel's claims of security, performance, and fitness for the purpose for which it was sold, which later was found to be false. They are being sued exactly the same way that Honda would be sued if they supplied a Formula One team with an engine that performed 30% slower than advertised.Alexvrb - Saturday, March 24, 2018 - link
It's a CPU vulnerability.willis936 - Friday, March 23, 2018 - link
The power consumption drop with performance is interesting. I'd be interested in seeing a comparison of efficiency pre and post patch. At a glance t's difficult to tell if efficiency has gone up, down, or stayed the same. I'm under the impression that this patch disables speculative execution entirely which, unless speculation takes a lot more power than I think, efficiency should go down.