Loading...

Tech Talk Live Blog

Spectre and Meltdown: A Sysadmin’s Perspective

Shawn Mellinger


The last two weeks the Internet has been buzzing with talk about the Spectre and Meltdown vulnerabilities. That is understandable given the wide range of computers that they affect. However, we have seen major vulnerabilities over the last few years, notably with the NSA exploits being leaked last year. In the past we have identified the vulnerable computers, patched them, and moved on with our jobs. So what is different this time?

For the sake of easy conversation I will lump the Spectre and Meltdown vulnerabilities together. These exploit critical vulnerabilities in most modern processors. So rather than finding a vulnerability in software, researchers have found the vulnerability in how processors and applications maintain isolation. I would recommend checking out this site which contains excellent information and videos of the exploits in action.

Given that there are both hardware and software issues, how do we protect our systems against these vulnerabilities? The good news is that there are patches available that modify the software issues and CPU microcode. The bad news is that it is not as easy as running Windows Update’s simple patch.

Intel was informed months ago and began coordinating with many major vendors including Microsoft and OEM computer manufacturers.  This allowed them to create patches to address their portion of the vulnerability. They were working together to announce the vulnerabilities and provide patches along with them. However, in late December some core information about the vulnerabilities leaked out which led to a rushed vendor response to the issue in early January. That rushed response has led to some issues that I will highlight later.

To fully patch there are three main areas that need updated: the CPU microcode, operating system, and applications (mainly web browsers). The CPU microcode update is largely handled by a BIOS update. The operating system such as Windows has updates released for most modern operating systems; however, with some complexity applying them. When Microsoft was testing their update they found that “some third-party applications have been making unsupported calls into Windows kernel memory that cause stop errors (also known as bluescreen errors) to occur.” Their response to the issue was to require antivirus vendors to set a registry key to show that they were compatible with the update. Without that registry key Windows Update would not show the update as available. If the system does not run antivirus then that key must be manually added before they can run the update. Once the update is installed there are registry keys that need to be set to enable (or disable) the update. There are additional keys that need set for Hyper-V and Remote Desktop servers. Then, once the operating system is updated the applications (mainly web browsers) need updated so that they cannot be used to attack the operating system.

This complicates things as there is not a single patch to run. Many sysadmins I know have a set it and forget it policy toward BIOS updates, so many computers are still running the BIOS version that they came in with. In this case a coordinated effort is required to remediate every level or the vulnerabilities still exist.

Let’s take a look at practical ways to get systems up to date. First, I will address the BIOS update. In my mind this is one of the trickiest to resolve. While safe, it is often impractical to walk around and manually update the BIOS on each computer. Dell and Lenovo offer client software to update BIOS; however, often that is removed while cleaning the master image. In my situation I think it makes the most sense to push out BIOS updates via a configuration management software such as SCCM. Communication with users will need to happen since a reboot is required, plus some careful planning on timing.

Windows Updates are the next logical step. Before we can download and apply the updates, that antivirus registry key needs to be set. The good news is that most major antiviruses have updated and have placed the key. This just leaves any servers that may not run antivirus for various reasons. A group policy or SCCM baseline could set this key, and the updates can be applied via your preferred method. Then the registry keys need to be enabled. For Windows Clients the registry keys are enabled by default, but can be disabled using registry keys. These keys are set as follows:

reg add “HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management” /v FeatureSettingsOverride /t REG_DWORD /d 0 /f

reg add “HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management” /v FeatureSettingsOverrideMask /t REG_DWORD /d 3 /f

You can also disable the keys by adjusting the registry as detailed in Microsoft’s guidance document. For Windows server there is slightly different guidance. Windows Server does not automatically enable those keys and will require them to be set. In addition, there is another key that needs to be set.

reg add “HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management” /v FeatureSettingsOverride /t REG_DWORD /d 0 /f

reg add “HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management” /v FeatureSettingsOverrideMask /t REG_DWORD /d 3 /f

reg add “HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Virtualization” /v MinVmVersionForCpuBasedMitigations /t REG_SZ /d “1.0” /f

You should test your workflows by setting the key manually for CPU intense operations. There was talk of a 30% performance hit by disabling speculation; that seems to be overestimated based on actual performance data. However, you should test as you can. Otherwise these keys could be set via group policy. Hyper-V hosts must shut down the clients entirely then start them again. This will resolve the CPU microcode portion of the attack.

Finally, updating web browsers can be handled via your configuration management of choice. The major web browsers to patch are covered by the latest versions such as Firefox and Chrome, as well as IE and Edge (addressed as part of the Windows Update).

VMWare also has provided updates to resolve the vulnerabilities. However, remember how I said the response was rushed? Intel has worked with several vendors to pull back their BIOS patches as they still contain flaws. VMWare was forced to pull back their patches and continue to work with Intel to resolve the issues. As of this article no new patches have been released to fully resolve the issue. Similarly, HP removed their patches as well.

So far the best way to check Windows systems to ensure full protection is to use the Microsoft provided tools. They have released a PowerShell module which makes it simple to check the system which can be run using the following.

PS > Install-Module SpeculationControl

PS > Get-SpeculationControlSettings

However, this module requires PowerShell 5 which is only loaded by default on Windows 10 and Server 2016. The command also does not provide any remote capabilities, and uses Write-Host to display data, so writing to a log is hindered. For a larger environment, Microsoft released an SCCM Compliance Baseline. This provides nice reporting to see how much of your environment is protected. However, it does not provide remediation for servers which do not contain the registry keys to enable the fixes, which would be nice.

Even though this has been a hot topic over the last few weeks, it continues to evolve as manufacturers find out more. As of right now the BIOS update portion of fixing the vulnerabilities looks to be a changing game that may not be completely sorted. We may see other vendors pull back their microcode patches as Intel reviews the changes. However, the browser and operating system updates appear to be stable and should be applied. This blog is intended to give you an overview of the situation as it stands now, but I would love to hear from anyone who has better ideas on how to patch the vulnerabilities.

Tech Talk Live Blog Comment Guidelines:

One of our main goals at Tech Talk Live is to build a community. It is our hope that this blog can be a forum for discussion around our content. We see commenting as an integral part of this community. It allows everyone to participate, contribute, connect, and share relevant personal experience that adds value to the conversation. Respect counts. We believe you can disagree without being disagreeable. Please refrain from personal attacks, name calling, libel/defamation, hate speech, discriminatory or obscene/profane language, etc. Comments should keep to the topic at hand, and not be promotional or commercial in nature. Please do not link to personal blog posts, websites, or social media accounts that are irrelevant to the conversation. This is considered self-promotion. We welcome links that help further the conversation and reserve the right to delete those we deem unnecessary. The appearance of external links on this site does not constitute official endorsement on behalf of Tech Talk Live or Lancaster-Lebanon Intermediate Unit 13. You are solely responsible for the content that you post – please use your best judgment. We reserve the right to remove posts that do not follow these guidelines.

Leave a Reply

Your email address will not be published. Required fields are marked *

CONTACT

Tech Talk Live is the only conference of its kind in the region specifically designed for IT pros in education.


techtalklive@iu13.org
1020 New Holland Avenue, Lancaster, PA 17601

(717) 606-1770