Prioritizing and remediating vulnerabilities in the wake of Log4J and Microsoft’s Patch Tuesday blunder

The past few weeks left IT professionals overwhelmed as organizations scrambled to assess if they were vulnerable to threats posed by the Log4Shell vulnerability. As if that weren’t enough of a challenge over the holidays, more Log4j CVEs followed, not all of which deserved equal attention.

And Microsoft’s January Patch Tuesday flaws caused even more confusion, with the first batch of updates breaking functionality, forcing another round of updates.

Such is the predicament often faced by IT and cybersecurity professionals: Figuring out which vulnerabilities are most critical and deserve immediate attention, what can wait, and when to trust and apply an update.

The skill requires sharp technical understanding, planning, and tact as delaying crucial patches, under the false assumption that the vulnerabilities they cater to pose a limited threat, can cause devastating consequences. On the flip side, dedicating resources to patching flimsy “vulnerabilities” that may never be exploited without attackers gaining administrative access to a system could become a wasted effort.

The CVSS scoring model does not measure risk

Introduced in 2005 after research by the National Infrastructure Advisory Council (NIAC), the Common Vulnerability Scoring System (CVSS) standard was designed to help organizations assess the “severity” of the flaws. While CVSS scores provide a decent idea of a vulnerability’s severity, they can fall short. CVSS 3.1 scoring guidelines emphasize that CVSS scores are designed to measure a vulnerability’s severity and not the risk it poses.

When scoring vulnerabilities, the analyst is advised to “score for the reasonable worst-case implementation scenario,” but it isn’t uncommon for two skilled cybersecurity analysts to arrive at different scores, making the process subjective. A “critical” vulnerability might just as well be scored as “high” depending on to what extent can an analyst articulate the realistic possibility of the worst-case scenario existing in an environment.

CVSS scores are subjective and often change

We noticed this in the case of Log4j CVEs. Of the five CVEs released thus far, including Log4Shell (CVE-2021-44228), the severity of two vulnerabilities was adjusted as more information was discovered.

Previously thought to be a “low” severity DoS flaw scoring a 3.7, CVE-2021-45046 was later bumped up to a “critical” (9.0) rating. Whereas, severity for another DoS flaw (CVE-2021-45105) discovered subsequently was downgraded from “high” to “moderate”.

Ax Sharma

Of all the CVEs released for Log4j, however, the disclosure of the last remote code execution bug CVE-2021-44832 drew sharp criticism from the infosec community given the flaw’s exploitation required the attacker to already have write permissions to the Log4j logging configuration file and intentionally make the instance vulnerable. Some questioned if a CVE assignment for this issue was warranted in the first place, especially at a time when the understaffed Log4j team of volunteers and users was already swamped over the holidays patching legitimate Log4j flaws.

“With Log4j, the only vulnerability that mattered was the original one: CVE-2021-44228,” CERT Coordination Center’s vulnerability analyst Will Dormann tells CSO, referring to the fact that Log4Shell was being actively exploited in the wild.

Be cautious with follow-on reports of vulnerabilities

“Then came the CVE pile-on. Other organizations, researchers, etc. decided that it’d be neat if they found their own Log4j vulnerabilities. But out of each of the follow-up vulnerabilities (CVE-2021-4104, CVE-2021-45046, CVE-2021-45105, CVE-2021-44832), successful exploitation required that the attacker already have control over the Log4j configuration file… and/or the software using Log4j is using it in a contrived way that isn’t even seen by any software in the wild,” explains Dormann.

Following disclosure of a critical vulnerability or exploitation technique, it isn’t uncommon for other bug bounty hunters to be motivated and shift their focus toward finding similar flaws—to enrich their CVE portfolio or collect bounty prizes. A similar trend was seen following the disclosure of dependency confusion technique and the many copycat packages that emerged, continuing to infiltrate open-source repositories today.

However, because open-source projects are often run by small teams of volunteers who are supporting critical projects in their spare time, unnecessary feature requests or vulnerability reports for flaws that are abstract proofs-of-concept and practically unexploitable under normal circumstances can place additional, undue burden on the maintainers.

Apache’s Position Paper, drafted January 2022, advises that “security directives MUST avoid placing additional unfunded burdens on the few maintainers who are already doing the work.”

It is not just the project maintainers who are at a disadvantage following the disclosure of a series of high-profile vulnerabilities: security and IT teams across organizations using the products will now also have to spend resources on assessing the impact from each of the vulnerabilities and prioritize patching activities.

Pointing specifically to CVE-2021-45046, after its CVSS score was revised to a 9.0, Dormann cautions, although the higher severity score may raise alarms since the flaw’s disclosure from over a month ago, “nobody on the internet has been able to come up with a single real-world application that is affected by it. Yes, some vendors who didn’t do their homework probably published an advisory saying that they were affected by it but didn’t actually confirm that their software was actually affected,” he continues.

Dormann argues without knowing the context in which a component is used in your application (e.g., a specific vulnerable class or method from the component), merely declaring an application vulnerable simply because it has a certain Log4j version could be inaccurate. “If a CVE gets a CVSS score, that score may not even accurately reflect the severity of that instance of the library in any given product.”

Microsoft’s patching blunder makes admins wary of updates

While the industry was starting to recover from the Log4j episode, January’s Patch Tuesday left sysadmins confused and displeased. On January 11, Microsoft released a series of Windows Server updates that caused issues with Hyper-V virtualization instances and ReFS file systems.

Shawn Waldman, CEO of Secure Cyber Defense, tweeted he was “denying these patches due to constant reboots.”

Adriano Maia, CISO at Proteus Risk Management, didn’t hesitate to voice his concerns either: “Did MS even [test] this wave? Anybody else having problems out there? From lsass crashing due to lsadb.dll to [Hyper-V] hosts going nuts.”

As a result, within two days Microsoft had to revert these updates due to the critical bugs and reissue them. Sure enough, this caused a great deal of confusion with regard to prioritizing and applying periodic updates.

This mishap was exacerbated by the fact that January’s Patch Tuesday fixed critical CVEs including ​​CVE-2022-21907, a remote code execution vulnerability in the Windows HTTP Protocol Stack, that Microsoft urged everyone to patch ASAP. “CVE-2022-21907 is a particularly dangerous CVE because of its ability to allow for an attacker to affect an entire intranet once the attack succeeds,” explains Danny Kim, principal architect at Virsec.

“Microsoft has stated that this vulnerability is wormable and should be patched immediately. The CVE is the latest example of how software capabilities can be warped and weaponized. The CVE targets the HTTP trailer support feature, which allows a sender to include additional fields in a message to supply metadata, by providing a specially-crafted message that can lead to remote code execution,” says Kim.

What can organizations do?

The challenge is no longer in just keeping up with applying the latest updates. If there is one thing high-profile supply-chain attacks like SolarWinds have taught us, it’s that not all updates are benign and can cause issues. Furthermore, the sabotaged updates of popular “colors” and “faker” libraries, and a legitimate dependency update that broke Facebook’s Create React App framework for many, are further examples of how blindly trusting and downloading the latest updates can break your software rather than fixing bugs.

Organizations must consider two critical aspects before deciding to act:

1. Of all the vulnerabilities (CVEs) disclosed, which ones deserve utmost attention?
2. Should the remediation for the most urgent vulnerabilities require downloading updates, are these updates legitimate, safe, and guaranteed to fix the problems?

Which vulnerabilities to prioritize?

When prioritizing CVEs, looking at their CVSS scores is a start. However, analysts need to evaluate the worst-case scenario posed by the hypothetical exploitation of a CVE in their environment. Flaws that can be exploited remotely by unauthenticated actors are typically almost always deemed a “high” or “critical” severity and should be prioritized especially if the vulnerable component is publicly accessible in your implementation.

For the remaining flaws — rated moderate, for example — if their exploitation is solely dependent on the exploitation of another flaw (e.g., as part of a vulnerability chain) or requires elevated privileges on the attacker’s part, their patching could likely wait for a bit, and the focus should be on assessing and eliminating the initial attack vectors that can aid in the exploitation of such flaws.

With a logging framework like Log4j, it becomes much harder to assess the impact from exploitation. This is because, for example, even if the logging server where the vulnerable Log4j version is running isn’t directly internet-facing, the webserver logging its activities, requests and inputs — legitimate and malicious — is enough to trigger the vulnerability even on a logging server behind a perimeter.

As such, even if at first glance the Log4Shell vulnerability doesn’t seem exploitable in your environment, a thorough assessment of perimeter controls and where the vulnerable component resides on your network might help paint a better picture.

The Stakeholder-Specific Vulnerability Categorization (SSVC) framework from CERT/CC aims to provide a better picture to analysts when evaluating how applicable a vulnerability is to their environment by trading a one-size-fits-all for a modular decision-making system that has clearly defined parts that vulnerability managers can pick and use as appropriate to their context.

“SSVC is about reliably converting evidence into decisions during vulnerability management… and aims to be a shared language between decision makers and analysts, so leadership can make risk-based decisions and the analysts can consistently gather the evidence and execute on those decisions,” explains Dormann.

Although SSVC by itself may not be sufficient to address all questions, such as scoring vulnerabilities in a library like log4j, the framework helps with decision-making when prioritizing what vulnerabilities to cater to.

As a side note, being in touch with a vetted community of infosec professionals can be quite rewarding here, and save time, even over social media. Subscribing to infosec digests and open-source project mailing lists can provide timely intel from qualified and credible professionals who may be able to give a heads up on what vulnerabilities are critical among the noise. Granted, any such correspondence may need further vetting by your organization’s in-house teams, it can at least point your analysts in a direction worth looking into.

To update or not to update?

The second aspect of verifying and vetting updates is a bit more challenging and requires both manual intervention and automation. While downloading and applying the latest patch for a vulnerable component is the recommended guidance for many vulnerabilities, workarounds may also be provided. Decision makers must evaluate which option provides a better outcome: patch or manually apply a workaround.

If the benefits provided by applying the workaround outweigh the costs and potential risks posed by a downloading a possibly suspicious update, it may be worth it to just manually apply a workaround. There is a chance of breaking something by updating a configuration file, but the risk is relatively manageable and controlled than the risk from downloading an unvetted update, as in the cases of the aforementioned examples of SolarWinds and dubious dependencies.

Furthermore, using automated  SCA tools and Log4j scanners, such as those released by CISA, Google and Microsoft (in Defender 365), can spot the specific vulnerable code snippets and classes are lurking in your applications. The deep binary scanning and string-matching approaches used by these tools may successfully report the vulnerable classes instantly rather than manually having to reverse engineer binaries and determine if they are affected.

On the other hand, buggy patches like the ones in Microsoft’s January Patch Tuesday are much harder to predict in advance and luckily seem to be a rare instance. The problems caused by these updates may only be discovered via infosec community members, mailing lists, and social media channels—after someone has already been affected by them.

Microsoft did release emergency out-of-band (OOB) updates to fix the issues caused by the buggy Windows Server updates, but that still doesn’t resolve the initial confusion caused by the broken updates in the first place, and the dilemma: Jump on Patch Tuesday updates right away or test them in a lab environment. Another approach is to rely on proactive monitoring solutions that can provide some protection in absence of patches.

“Although Microsoft has provided an official patch, [CVE-2022-21907] is another reminder that software features allow opportunities for attackers to misuse functionalities for malicious acts,” said Virsec’s Kim. “Instead of trying to continuously patch and identify these vulnerabilities, enterprises should look for a real-time monitoring solution to safeguard applications and their functionalities from these types of attacks.”