The killer app for mashing malware

Most successful attacks on client desktops occur when end-users are duped into launching Trojan horse executables, a fact I’ve raised time and again in the past couple of years. Users might think they’re installing an Outlook security patch, a recommended anti-virus program, or codec needed to watch Britney slink out of a cab, but they end up unleashing malware that can wreak havoc not only on their systems, but on your entire corporate network.

Unfortunately, no matter how up to date your organization’s anti-virus software is, there is no 100 percent effective way to stop this type of attack once a user opens the infected file. They’ll work regardless of whether you’re using Windows — the most targeted platform — or OS X, Linux, or BSD.

[ Learn why the InfoWorld Test Center says malware-fighting firewalls miss the mark. | Mobile malware is cropping up at an alarming rate. ]

For years now, anti-malware companies have tried everything to combat malware, but without great success. Part of the problem is that anti-malware software is struggling as never before to detect the tens of thousands of new variants being generated daily by criminals and hosted on legitimate (or legitimate-looking) Web sites. The body of evidence shows that the best detection rates are between 40 and 70 percent — and most products are not on the high end. (Here’s a good report on the subject.)

Compounding the problem is the inherent shortcomings of today’s anti-malware software: Installation and execution warnings tend to be non-existent, overly generic, or excessively enthusiastic, eventually teaching users to ignore them with a click.

It’s time for the anti-malware industry to completely reinvent how we detect malware and put the intelligence of the professional anti-malware engineer into the hands of every end-user. Security software developers need to take the best of all the various technologies (signature detection, heuristics/behavior detection, whitelisting, blacklisting, code signing, community groups, and so on) and make them work in concert to give end-users the best chance of escaping infection.

Anti-malware designed for end-users

If I were an anti-virus software developer with sufficient resources, here’s how I’d design an anti-malware product for analyzing files downloaded or executed for the first time by an end-user. Essentially, each file would go through a series of steps to determine if it’s good or bad. If the software can’t determine the file’s status at a given step, it is subjected to further examination.

First, the software would probe the executable for digitally signed code, using a certificate from a trusted certification authority. If the code were signed (identifying the developer/signer), installation or execution would continue — whether performed manually or automatically from a Web site — with the user’s blessing. This probably would eliminate the need to scan most of the very popular software, such as products from Microsoft, Adobe, Macromedia, and so on.

If the code were unsigned, the software would consult a local copy of a community-based whitelisting database containing file hashes of the most popular, secure unsigned executable files. If the file’s hash was listed in the whitelist database, installation would proceed without further warning or testing. This local database would contain perhaps a few thousand entries and would be updated frequently to contain hashes of increasingly popular files.

If the code’s hash file didn’t show up in the local whitelist database, this security software would employ anti-malware signature detection to examine the executable. If the executable passes signature detection and is not deemed malicious, the anti-malware software would connect to a larger, global, community-based, whitelist database distributed on the Internet (assuming Internet connectivity is available). This database could contain millions of hashed entries. The anti-malware client could use one-query-packet-sent, one-answer-packet-received-type queries (a la DNS) to get quick answers.

If the inspected file was not located in the global whitelist database, it would be analyzed with a local heuristic/behavioral analysis engine, optimized to detect suspicious behaviors. The found behaviors can be ranked with weight-averaged scores, the way anti-spam software detects e-mail for suspicious traits. If an executable were to exceed a certain scoring threshold, the malware software would mark it as malicious. If it is ranked below a certain score, it would be declared most likely safe.

In the event that the suspected file’s ranked score could not definitively determine the code’s status, the user could opt to halt installation and submit the file over an encrypted tunnel to a trusted security community for tool and human analysis. Once the community has examined the file, it would return its answers to the originator and store the result in the online database. Frequent queries about the same file get pushed locally.

Finally, failing any definitive confirmation, the user would be told that the file’s level of risk could not be determined. It would then be up to the end-user to decide whether to install the file. This is essentially what happens most of the time without all the analysis I just talked about. In my perfect world, the untrained end-user would first rely on smarter software to assess the potential danger of a file before opening it.

Whitelisting comes to the fore

The idea here is to use the various anti-malware technologies where they make the best cost/benefit sense in the inspection pathway. We could eliminate a lot of anti-malware signature-detection slowness by utilizing whitelisting. As of last year, more malicious programs were made than legitimate, making whitelisting potentially more accurate than blacklisting (for example, anti-malware signature detection). By themselves, none of the anti-malware technologies are even close to perfect. But when efficiently organized and working in concert, these technologies would make a vastly improved security app. Any takers?

Related content

Malware-fighting firewalls miss the mark

InfoWorld Test Center attacks Astaro, SonicWall, WatchGuard, and ZyXel firewalls, and only one puts up a fight Starting from scratch is the only malware cure

If you discover malware on your system, don’t mess around. Back up your data, format your hard drive, and begin again

Fighting malware: An interview with Paul Ferguson

Roger Grimes talks with security expert Paul Ferguson about the present and future of malware

InfoWorld’s Malware technology channel

Discover the latest news and reviews on all things malware