The Security of Open Source Software

Open source, as used today, is not necessarily more or less secure than proprietary closed-source solutions. However, with automated program analysis tools, open source has the potential to be dramatically more secure than its commercial alternatives.

In November 2003, a malicious hacker tried to compromise the Linux operating system—not a particular computer running Linux, mind you, but the whole thing. Here's how: After taking over a series of computers at an undisclosed university, the individual (or individuals) penetrated a server used by the Linux development team. Once there, the person inserted two lines of code—a so-called back door—into the very source code that is used to compile the Linux operating system.

This back door was quite elegant. To exploit it, all anyone would have to do is run a two-line program and—wham!—the attacker would instantly have his privileges upgraded to "root," the Linux equivalent of the Windows System Administrator. Essentially, the hack would have made it easy for an attacker to escalate his privileges. Had the code been compiled and distributed, the implications could have been far-reaching.

Of course, this hack was not to be. The attack was discovered less than 12 hours later when automated tools used by the Linux developers detected the unexplained discrepancy. The code was quickly removed, and no computers were jeopardized.

What's so maddening, from the perspective of today's CSOs, is that it is theoretically impossible to look at any piece of sufficiently complicated code and tell for sure if it has a security vulnerability. In fact, it's even impossible to determine if an intentional back door has been added to a program. The problem isn't that terms such as "vulnerability" and "back door" aren't well-defined. The problem is that programming languages are too powerful: It is possible to so completely hide functions and features inside a program that the only way to find them is by running the program itself—and then it's too late!

The good news for CSOs is that if you are willing to settle for less-than-perfect security, then many common programming flaws and even intentional back doors can be readily detected with a new generation of automated program analysis tools. The tools will help you find vulnerabilities, and some even perform a risk-benefit analysis to see if the vulnerabilities are worth fixing.

Planted Attacks

The Linux attack demonstrates a very real risk in today's open-source software: Because the software is by definition distributed in source code form, it's quite easy for an attacker with even relatively modest skills to plant a malicious attack.

Indeed, it's happened before.

Six years ago, a hacker broke into a computer in the Netherlands that was the distribution location for a firewall toolkit called "TCP Wrappers." Once again, a back door was added. But this time, the vulnerability was put into a piece of code that was being actively downloaded and deployed. Between 7:16 a.m. and 4:29 p.m. on Jan, 22, 1999, a total of 52 sites downloaded the compromised program. Some may have even installed it. The author of the program, Wietse Venema, discovered the unauthorized alteration and stayed up into the early morning notifying all of the affected sites. He then wrote a disclosure for the Computer Emergency Readiness Team at Carnegie Mellon University, which published the alert the following day.