New approaches to recognizing signature-evading, polymorphic viruses, worms, and bots
Server-side polymorphism is a challenging problem for anti-malware software vendors. Much of today's malware, such as the Storm worm, creates tens of thousands of variants each month, a development that has made many anti-virus software programs that use static signatures significantly less accurate.
Many of today's worms and bots, which make up a large portion of the malware out on the Internet, launch from innocently infected Web sites. The criminal (or the criminal's automated program) breaks into the innocent Web site and injects code that then tries to infect every visitor running unpatched software. If the visiting user is fully patched, the site will prompt the user to install a "needed" program. The users, not realizing that their favorite trusted site is infected with malware, often follow these instructions.
The first software that is downloaded to the user's machine and executed is a small malicious program with only one intention: silently install more malicious programs. The "downloader" installs anywhere from a handful to dozens of other malicious programs, each coming from a different infected or malicious Web host on the Internet. The eventual client-side program is intended to steal money (one way or another) or to be used in a distributed denial of service attack.
The programs and scripts used in these malicious exploits are often unscannable by traditional anti-virus scanners. The malware is not recognized by the signature recognition engine of the anti-malware program because the program never existed before. It's a "one-off." Every separate user connection to an infected Web server results in a different version of essentially the same malware program.
Mutating malware
Variant and mutating malware is nothing new. It's been around for more than two decades. What is new is that instead of the malware program itself doing the mutating, the pseudo-polymorphism (it really isn't true polymorphism) is generated by the malicious server-side program. Until the user connects to the generating software, the malicious variant doesn't exist. The malicious coding on the server creates a copy of the malware program, but re-encodes or packs the program, using a different packing engine. Even before the packing, it changes the names of the internal variables used, along with subroutine names and program logic locations. Program instructions and subroutines get moved around, encoded with arbitrary ASCII characters, and even the malicious payload is created on the fly.
Who cares whether anti-virus companies get the malicious sample and make a signature? The malware has never existed before and won't exist again. Server-side polymorphism has created another challenge in the anti-virus world.
Previously, mutating viruses could be recognized by the internal coding used to conduct the mutating, some of which can't be hidden if the result is going to mutate properly. Anti-virus vendors figured out that they could forget trying to recognize the actual malware, which could take a billion different forms. Instead they just look for the code of the static mutation engine. Authors of polymorphic malware have tried to complicate the process by morphing the mutation engine signature, with varying degrees of success.
With server-side polymorphism, the mutation engine is left on the originating server. The malware doesn't have a static mutation engine component to identify in the first place. Add to this the fact that the mutating code is packed, and often re-packed again many times, and anti-virus scanners have their work cut out for them. It's no surprise that anti-virus software has been declining in accuracy against new, "one-off" threats. But like the AV vendors have done for 20 years against emerging threats, they are battling back.
VILO project
One approach is a technique used by the VILO Malware Defense Suite project. Started by a few researchers at the University of Louisiana, Lafayette, the Center for Advanced Computer Studies project has analyzed thousands of server-side polymorphic malware programs. They take each one-off and normalize the coding. They take what is essentially pseudo-random subroutines and names and pull out the patterns. Because most server-side polymorphic code is based on a very small subset of malware families, the researchers can essentially convert the random-looking code back into the original family member.
On the VILO Web site, a user can submit a malware sample to be analyzed. The incoming sample is compared to a large database of previously collected malware samples that have been normalized and analyzed. The commonalities and differences are noted and a match is made. The VILO Web site will tell you how likely the submitted sample is to previously analyzed and defined malware. Submitters can even compare assembly language binaries against each other, instruction to instruction, to see the comparison themselves.
All of today's popular anti-virus vendors are using similar techniques to recognize server-side polymorphic code, among their quiver of technologies. I like the VILO site because I can submit my own found samples that aren't readily identified by anti-virus software programs, and then compare the coding myself.
If you're into learning more about malware and anti-malware defenses in detail, I highly recommend the Virus Bulletin newsletter. Although a subscription can be expensive (starting at $175 per year), it's well worth the money. It's probably one of the most respected industry journals, and each month its 20 to 30 pages are chock full of useful information. Each issue contains two or three major articles, an opinion piece, good technical reviews of malware and anti-malware programs, and a solid anti-spam supplement. Most articles are written by highly skilled (and highly opinionated) anti-malware defenders. It's been around for many years, and I look forward to every issue.