Rating software security Consumer Reports-style

The Cyber Independent Testing Lab (CITL) is fuzzing binaries at scale and building a checklist of compile-time security best practices.

Become An Insider

Sign up now and get FREE access to hundreds of Insider articles, guides, reviews, interviews, blogs, and other premium content. Learn more.

The poor security of much enterprise software can be dramatically improved at low cost with the compile-time equivalents of seatbelts and airbags. With that in mind, the Cyber Independent Testing Lab (CITL) is building a Consumer Reports-style rating systems to grade the security of thousands of software binaries.

Founded by l0pht hacker and former head of cybersecurity research at DARPA Peiter "Mudge" Zatko, and bankrolled with seed funding from the US Air Force, the CITL presented their methodology and some preliminary results at the 34c3 hacker conference in Leipzig, Germany a few weeks ago.

"It's ridiculous," Tim Carstens, acting director of the CITL, says. "At the enterprise scale, you've easily got a hundred thousand different binaries running in different places, and some infinitesimal fraction of that has the latest security features, and most of it isn't compiled in a way that enables those trivial defenses."

While basic compile-time security features like ASLR or DEP may not be silver bullets, they do make an attacker's job much more difficult. The vast amount of low-hanging fruit that attackers currently enjoy can be taken away from them, and at low cost to software vendors and enterprise security administrators. "At scale a lot of really basic defenses are not present," Carstens says. "Major vendors' software does not have stack execution prevention or heap overflow prevention enabled, and this is software that has an attack surface."

To solve this problem, the CITL is building a checklist of compile-time security best practices. "For software vendors, the main question I would pose to them is, what is their pre-release process on their gold image?" Carstens says. "What is their prerelease checklist? Check for the presence of compiler hardening features like ASLR and DEP, things in that class."

To encourage vendors to prioritize security, the CITL is mass testing thousands of publicly-available binaries against their checklist, and plans to publish Consumer Reports-style ratings. Enterprise security administrators will be able to use the CITL's ratings to identify weaknesses in their infrastructure and to demand more secure software from their suppliers.

To continue reading this article register now

SUBSCRIBE! Get the best of CSO delivered to your email inbox.