How to Evaluate Software Security: Mary Ann Davidson

Q: Has the focus on identity theft taken attention off code quality?

A: Most theft of sensitive personal information that can lead to identity theft has been done through social engineering, or through trusted insiders, neither of which has anything to do with code quality per se. Of course, one of the many ways to steal data could be through exploitation of a security fault in software.

Both identity theft and poor code quality are horrifically expensive to those who experience them. The cost of “bad security,” for example, is estimated to be as high as $59 billion per year.

Q: What’s new in vulnerability scanning software?

A: There are many tools on the market: Some scan source code, some scan applications. Some tools look for known, specific vulnerabilities in products. Unfortunately, the false-positive rate on some tools is as high as 90 percent, which makes the problem worse, since you are spending scarce security resources chasing phantoms.

Source code scanning is one of the right problems to solve. Currently, these tools are widely available and proven to work with very low false-positive rates and high scalability. Some argue that it should become a procurement requirement that vendors run their products through a source code scan.

Q: How does Oracle ensure that the end product is secure?

A: We have many steps in place to build security into the development process, including:

* Secure coding standards with mandatory training

* Security requirements within functional, design and test specifications

* Security “exit criteria” for releases

* Security regression tests

We’ve developed in-house tools to scan code for certain classes of vulnerabilities (such as SQL injections), and we are looking at licensing additional automated tools.

We have an ethical hacking team to help break products. We also do formal security evaluations against the Common Criteria (ISO-15408) and the U.S. Federal Information Processing Standards (FIPS) 140.

Q: How should I evaluate Oracle’s products without access to your source code?

A: The best ways to do this is due diligence with your vendor and looking for third-party assurance (Common Criteria evaluations). Due diligence questions include:

* Do you have a formal development process that includes security?

* Are developers trained on secure coding?

* Is compensation tied to secure coding practices?

* Are products assessed by security experts (in-house or other)?

Common Criteria evaluations do not guarantee fault-free code; they do validate that the product meets its intended security claims.

Q: What are the best standards available to test code security before an application is released?

A: The automated tools available for testing security are not all quite “there” yet, but vendors should evaluate them anyway. Other methods can also test security. For example, there are tests that explicitly check that the security functions do what is expected and do not do what is not expected. Ethical hacking teams can help. Training QA people (as well as developers) to think like hackers so that they can break code also helps. Making all the above tests repeatable (as regression tests) helps too.

The evolving vulnerability management segment seems to be gaining strength. That is the ability to centralize and coordinate or correlate patches, events, vulnerability assessments and security configurations with security policies in an automated fashion. The truth is that nobody can keep up with security patches. Look for products that can analyze what you are running, the vulnerabilities in what you are running and the severity of those vulnerabilities as well as which systems you have that are most critical to patch first. This will help you develop a prioritized patching plan and make the best use of people’s time and money in applying patches.