How to Compare Patch Management Software

Patch management software helps organizations acquire, test and install code to fix known vulnerabilities in operating systems and applications. It also helps them assess exposure and prioritize patches (given your specific environment), identify missing patches that need to be remediated and produce real-time reports for compliance and other auditing needs.

Since its emergence early this decade, patch management has become “operationalized,” says Ronni Colville, an analyst at Gartner. For instance, the function is being subsumed into PC configuration management vendors’ suites, such as Symantec (Altiris) and Avocent (LanDesk). However, she says, in most cases, these systems don’t offer the richness of capability provided by point solutions.

Also see Patch Management Systems: Evaluation Criteria and Capabilities

Three main players remain in the point solution market: BigFix, Lumension and Shavlik. Still, Colville says, “no vendor can make a full business on just patch management, so they’ve brought in other functions.” For instance, BigFix has broadened its security focus to include more configuration functions (such as inventory and software distribution), she says, while Lumension and Shavlik have begun to include functions such as security configuration, endpoint vulnerability assessment, and data leakage prevention.

Meanwhile, some companies continue to use Microsoft Windows Server Update Services (WSUS) to patch Windows operating systems and applications because it’s free, but it’s also more manually intensive.

Patch Management Package Selection: Two Prime Considerations

Configuration management versus patch management. A primary decision is whether to turn to a configuration management system for its patch capabilities or to a point product that may or may not also offer configuration capabilities. According to Colville, the reasons organizations choose the latter is they’re not ready to commit to a full lifecycle configuration suite, or their current configuration management tools don’t provide a best-of-breed patch management capability. The trade-off of having both in your environment, of course, is the need to deal with multiple agents and consoles.

Eric Maiwald, an analyst at Burton Group, suggests first evaluating your configuration management system for its patch management capabilities, since it might be advantageous and less expensive to maintain the same architecture for both functions, especially if it already works well in your environment. However, if you change your mind, the functionality will be more difficult to remove because the single-vendor approach means the software is embedded more deeply into your architecture.

Agent versus agentless. As Shavlik explains, agentless systems are based on push technology and on a centralized design. Server-based software scans the machines in the enterprise and initiates all actions on those machines. With agent-based solutions, client-based software scans the machine and communicates its findings back to the central console.

Agentless systems are best for networks with large amounts of available bandwidth and connected machines. Agent-based systems are best for environments with frequently disconnected machines, such as mobile PCs, and distributed networks with remote locations that have limited bandwidth.

“With an agent-based system, you get more operational control and better ability to do inventory scanning and monitoring,” Colville says. “What you give up is ease of deployment and lower management effort, and you’ll pay a higher price.”

Dos and Don’ts of Patch Management

DO decide between agent-based or agentless. Mark Starry, director of enterprise architecture at Concord Hospital in Concord, N.H., decided on an agent-based system from BigFix in 2004 because he thought it would better fulfill his compliance needs. The system enables him to report in real time on which patches were actually installed, not just which patches were deployed. Although it’s unlikely, he would also be able to detect whether a user uninstalled a patch. He also found the BigFix agent to have a smaller footprint than other options at the time.

Starry also finds it useful that the agent can report on what software is installed on which PC, not just what is listed in the registry. “We know in a second which of our 5,000 machines are vulnerable, so we’re able to react that much more quickly,” he says. He can also scan the network and subnets to discover any rogue machines, since it’s the corporate standard for all PCs to have BigFix on them.

Meanwhile, at Tamiyasu, Smith, Horn and Braun Accountancy, Susan Bradley, a CPA who also oversees the firm’s computer systems, runs a small business server network with fewer than 50 desktops attached. She chose Shavlik’s agentless system because most people are attached to the network, and even people who access the network from home are not making a full VPN connection. “I don’t have to worry about laptops because they don’t have that much connectivity into the network to be a risk, and their patch status is irrelevant,” she says.

DO evaluate the vendor’s nonpatch capabilities. Patch management is only 40 percent of what BigFix does, Starry says. He also uses the BigFix system for endpoint protection, Windows firewalls, software asset management and power management.

But not everyone wants all those capabilities through a single system. Ray Jacob, director for network and systems management for the New York City Department of Housing Preservation and Development (HPD), likes that he has multiple engineers with different network skill sets. “The people I have running Cisco security are different from the people running McAfee, which is different from Lumension,” he says. “I like the fact that we have a mix of people who are all contributing here.”

DO consider your architectural requirements. Some systems operate better in highly distributed environments, on large networks or in small environments. At Concord Hospital, one BigFix server manages 4,700 PCs and 400 servers, Starry says, although the system can scale to 250,000 endpoints. Colville agrees that a strength of BigFix is its scalability. “It can play in very large or very small organizations.”

Meanwhile, at HPD, Jacob needed a system that could stretch across multiple distributed points. HPD was able to distribute Lumension servers in five boroughs in New York City, with the main server at the primary data center in downtown Manhattan. “Endpoints are able to pull from a distributed point that’s the shortest hop away and sometimes even in the same building as the PCs are located,” he says.

At Tamiyasu, Bradley liked the fact that Shavlik was “nimble and lightweight. Others are very enterprise-tailored,” she says. “I couldn’t dedicate the database and hardware resources required.”

DON’T expect to “set it and forget it.” According to Jacob, just because you hear the word “automation” doesn’t mean you can click “enable” and let the system do the rest. In addition to creating a careful and thorough test methodology, “you have to tweak, control and plan deployments and do compatibility testing,” he says. For instance, when you look at a report that shows a certain number of patches didn’t get applied, you need to see why that happened and then redeploy them. At HDS, one engineer is a dedicated Lumension administrator, managing the deployments, tests, feedback and remediation actions, Jacob says. “I would say 30 percent to 40 percent of his time is devoted to the patch management process,” he says. “It does require man-hours, as well as always keeping risk in perspective.”

DON’T overlook testing. The vendors perform some internal testing before bundling up and distributing patches. However, this is mainly focused on determining whether the patch breaks standard software and verifying that it does what it claims to do, Maiwald says. For example, Starry says, BigFix provides quality assurance on the patches before releasing them. “If a patch is issued on Patch Tuesday, it’s in our hands by midnight or 1:00 a.m. or 2:00 a.m.,” he says. Patch Tuesday is the second Tuesday of each month when Microsoft releases its patches.

This does not, however, take the place of regression testing you’ll need to do on-site, Maiwald says. “The vendor doesn’t test all the possible permutations of what is going to happen when it’s applied.” The stakes get higher as the environment grows. “It’s one thing to push a patch out to 10 clients, but it’s a bigger deal with 1,000 or 10,000,” he says. Each enterprise needs to determine the level of testing required for different situations, Maiwald says, as well as the level of change management needed.

After Jacob receives the monthly patch bundle from Microsoft via Lumension, there’s a 10-day compatibility testing process. During that time, the operating system engineer attends a Microsoft webinar and uses Lumension to evaluate the security bulletins that Microsoft releases against HDS’s environment. The engineer then determines which patches are most relevant to deploy and distributes those to engineers in several test environments. The test environments are given five days to respond with the risk the patch imposes. A red signal means the patch broke something in the environment, yellow means there are warning signs and some support is needed, and green means the patch can be sent out in a general deployment.

Also see Broken Windows Revisited: Why Insecure Software Hurts the Global Economy

Even then, Jacob says, they don’t deploy to all 2,600 PCs at once. “For the first week, we might do 200 a night,” he says. “We’re still hedging because the testing might have missed something.” It’s important, he says, to minimize and control the impact so you can remediate quickly and continue your security operations without being too disruptive to the business. “It’s amazing how easily things can break,” he says.

Once all this is finished, Jacob says, it’s almost time for Microsoft’s next Patch Tuesday.

DO look beyond Windows applications. According to SANS Institute, unpatched desktop applications such as Adobe Acrobat Reader, Microsoft Office and Apple QuickTime pose a bigger threat to organizations than missing operating system patches. On average, major organizations take at least twice as long to patch client-side vulnerabilities as they take to patch operating system vulnerabilities, SANS says.

“Just about every software application may require patching, either for security or a functionality update,” Maiwald says. The question is whether it’s sufficient to rely on users to do that individually or if you want something more centralized or controlled.When Starry first implemented BigFix, it was primarily for Windows; however, “since then, my criteria have changed—I want to patch everything,” he says. Using BigFix, he can integrate Java, Acrobat, Flash and custom patches into his patch management process. Similarly, Bradley patches not just Windows but also Exchange and SQL Server.

DO check into ease of use. Given that patch management can be time-consuming, it’s important for it to be easy to administer, Jacob says.

“We needed the software to be easy from an administrative point of view from establishing flexible deployment windows,” he says. “If we needed to cancel or pause part of the deployment on a dime, we wanted to be able to do that easily and quickly.”

Bradley appreciates the ease of reporting in Shavlik. “You don’t have to reinvent the wheel,” she says. “When I’m scanning, it builds charts showing me what is most at risk. I don’t have to spend time customizing the reports.”