Home » Data Protection » Malware/Cybercrime

In Brief

How to Prevent, Detect or Remove Rootkits

What you need to know about rootkits

By Deborah Radcliff, Deborah Radcliff

November 01, 2006 — CSO —

Rootkits sneak in under the radar of computer security, hook deep into the operating system, then add malicious programs. They arrive via clicked-on links in e-mail, instant messages and websites.

Increasingly, organized criminals use rootkits to spread remote control ware (also called botware), spyware, spamware and keystroke loggers. They were present in 14 percent of the 5.7 million computers scanned by Windows Malicious Software Removal Tool, according to a June Microsoft report.

Rootkits start as low-level programs, such as Web helper applications, that are too small for security software to notice. Then they compile and open a back door to other programs that use the computer to relay e-mail and IM spam, or steal personal and regulated information. "Rootkits demand a new type of technology that finds and eliminates well-hidden malware. It's a much bigger job than antivirus companies can do," says Alan Paller, research director at SANS.

Emerging rootkit detection and removal tools are immature, each using different techniques. For example, some vendors like Websense and Sana Security claim to catch rootkits by looking for behaviors indicative of hidden malware operating in the background—such as servers initiating network calls, desktops talking to each other, or packets fragmented and transported inside and out of the network, says Peiter C. "Mudge" Zatko, technical director at BBN Technologies. Trend Micro released a consumer product that hooks as deep into the operating system as rootkits do to detect them.

But rootkit detection tools are each able to find only certain types of rootkits (for example, kernel-level or memory-hidden). Ask: Can a tool stop a rootkit from installing? Can it detect custom or targeted rootkits? If yes, can it remove them?

Update browser patches, and layer your security to include behavior and rootkit-level analysis technologies for protection, experts say.

Rootkits, meanwhile, are now hiding in virtual machine (VM) configurations, used to create virtual networks on a single machine for testing. At August's Black Hat Security Conference, researcher Joanna Rutkowska demonstrated how to use VM Ware to install a rootkit on Microsoft's new Vista operating system. Her suggestion: Restrict VM mode to only those computers that need it for development and research.