Software Patching: Patch and Pray

Patching-the only way to prevent poorly designed software from breaking everything-no longer works. And there's nothing you can do about it. Except maybe patch less. Or possibly patch more.

Then again, companies take more than a week to stick a service pack into a network. After all, single patches require regression testing and service packs are hundreds of security patches, quality fixes and feature upgrades rolled together. In a crisis, upgrading a service pack that was days old wasn't reasonable. Cooper soon learned that Best Software's MAS 500 accounting software wouldn't run with Service Pack 3. MAS 500 users who installed SP3 to defend against Slammer had their applications fall over. They would have to start over and reformat their machines. All the while everyone was trying to beat Slammer to the workweek to avoid a severe uptick in Slammer infections when millions of machines worldwide were turned on or otherwise exposed to the worm that, over the weekend, remained blissfully dormant.

"By late Sunday afternoon, Microsoft had two rooms set up on campus," says Cooper. "Services guys are in one room figuring out what to say to customers. A security response team is in the other room trying to figure out how to repackage the patches and do technical damage control.

"I'm on a cell phone, and there's a guy there running me between the two rooms." Cooper laughs at the thought of it.Repeat MistakesAs the volume and complexity of software increases, so does the volume and complexity of patches. The problem with this, says SEI's Hernan, is that there's nothing standard about the patch infrastructure or managing the onslaught of patches.

There are no standard naming conventions for patches; vulnerability disclosure comes from whatever competitive vendor can get the news out there first (which creates another issue around whether vendors are hyping minor vulnerabilities in order to associate themselves with the discovery of a vulnerabilityyet another story for another day). Distribution might be automated or manual; and installation could be a double-click .exe file or a manual process.

Microsoft alone uses a hierarchy of eight different patching mechanisms (the company says it wants to reduce that number). But that only adds to more customer confusion.

"How do I know when I need to reapply a security roll-up patch? Do I then need to reapply Win2K Service Pack 2? Do I need to re-install hot fixes after more recent SPs?" Similar questions were posed to a third-party services company in a security newsletter. The answer was a page-and-a-half long.

There's also markedly little record-keeping or archiving around patches, leaving vendors to make the same mistakes over and over without building up knowledge about when and where vulnerabilities arise and how to avoid them. For example, Apple's Safari Web browser contained a significant security flaw in the way it validated certificates using SSL encryption, which required a patch. Every browser ever built before Safari, Hernan says, had contained the same flaw.