January 2007 – Operating System Vulnerability Scorecard

Starting today, I plan on posting a monthly vulnerability scorecard for common server and workstation Operating System (OS) products.  I’m going to keep these scorecards pretty clean of discussion, but you can review my methodology, sources and assumptions on this page.  When folks have interesting feedback, comments or questions, I’ll consider starting separate posts for discussion and those can become references for future scorecards.

For workstation OSes, the product vulnerabilities analyzed include those applying to Windows Vista, Windows XP SP2, a subset of Red Hat Enterprise Linux 4 WS (rhel4ws), a subset of Ubuntu 6.06 LTS, and Mac OS Xv10.  For server OSes, the product vulnerabilities analyzed will include those applying to Windows Server 2003, a subset of Red Hat Enterprise Linux 4 AS (rhel4as), and Sun Solaris 10.  Note that the analysis for the Linux distributions excludes many optional packages in order to define more comparable product builds.  See Methodology, Sources and Assumptions for more details.

For each of the server and workstation OSes, the charts use a stacked barchart with highest severity vulnerabilities on the bottom and lowest severity on the top.  This allows an easy visual comparison if readers just want to compare just High severity, High + Medium severity, desiring to exclude lower severity vulnerabilities from comparison.

Workstation OS Vulnerability Charts

By workstation OS, I mean an operating system product that forms the basis for a computer users normal day-to-day computer-based activity, such as is comparable to Windows XP or Mac OS X, including a graphical windowing system and Internet browser, but excluding higher level applications such as Word, Excel or Powerpoint (which do not ship with Windows).

The first chart represents the total High, Medium and Low severity issues fixed for the various products over the past 3 months, ending in January 2007. Note that Windows Vista has only been available to business customers for 2 of those 3 months, having been released at the end of November. Examining the 3-month chart, we see that the Windows OS had the lowest number of total and High severity vulnerabilities fixed.

Next to get a view of 2007 year-to-date, we have a chart that just includes the vulnerabilities fixed for the products during January 2007. (In the next scorecard post, it will include January and February)

The results are largely self-explanatory, but I will note that for those that contend the Low severity issues for a product might not matter, one can exclude either the green or green and yellow portions as desired.

Server OS Vulnerability Charts

For server OSes, I am considering products that form the basis for a server in the network that would not typically be a day-to-day workstation for an individual user.  This means that, where possible, it is assumed that an administrator would choose not to install optional components like the graphical windowing system, Internet browser and so on.  On Windows Server 2003, those components are counted, since the user does not have an option to not install them.

Examining the 3-month chart, we see that, similar to the Workstations, the Windows platform has had to fix less total and less High severity issues than the other platforms.

 Next, looking at how 2007 is starting off for Server OSes, we see similar results to the 3-month view, but without the smoothing that as time provides to cumulative totals over longer periods of time.

 Vulnerability != Risk

Security professionals will correctly note that vulnerabilities represent only part of the security picture, with the risk equation also needing an understanding of the potential threats and value of the information at risk.  However, number and quality of attackers are elements largely orthogonal to factors that vendors have ability to influence.  Vulnerabilities, on the other hand, are a factor that vendors can influence directly by investing in process, testing and other best practice Q&A techniques to reduce bugs and raise quality of shipping products.

To put it into user terms, imagine that you are a CSO tasked with protecting some valuable company information on a company server.  You assume that the information is the target and that potential attackers will attempt to attack whichever platform you select to host the information.  In that case, the threat and value of the information is fixed, and the risk equation depends primarily on the vulnerability of the system you select (until you implement further mitigating actions).

Regards ~ Jeff