More secure software brought to you by the acronyms WRT and SQR

With breaches ever on the rise and software vulnerabilities at the heart of many security incidents, CSOonline decided to talk with noted software security expert Rafal Los. Los, currently security evangelist with Hewlett-Packard Software, is an industry veteran who has worked as a security consultant and even as information security officer in the Fortune 100. We wanted to get his thoughts on what organizations can do — today — to improve the security of the applications they develop.

Also see: Secure coding: A survival guide

CSOonline: There have been numerous articles, books and presentations on how to get a web application security program started. But there are very few on how to manage it properly once it’s in place. So once you’ve made the commitment, and have started down a secure software development path, how do you keep it on track?

Rafal Los: Once you’ve decided that you’re going to have a secure application development program, you’ve got to make sure that you’re measuring the right things. That is critical. I don’t think enough organizations do this. I think they just dive in and think: “Whatever we want to do is going to be great. People are going to nod in agreement, and there will be great fanfare and cheering and lots of excitement about application security.” That’s just unfortunately not the case. And organizations have to understand that measuring the wrong things can be just as bad as not measuring anything.

You mean measuring the wrong outcomes could hurt your efforts?

You could actually make things worse. With security, it can very quickly turn into a never-ending money vacuum. And what CIO’s fear very much is that’s exactly what it turns into: a never-ending money vacuum. It’s not supposed to be: “OK. Just keep spending money. There’s always risks to lower.” Well, of course there are always risks. But you have to make sure you are spending money wisely.

Get your morning news fix with the daily Salted Hash e-newsletter! Sign up today.

You have to be reasonable, right? If you get in the car, you put on your seatbelt. Now, there’s always risks that you could get in an accident where the impact is so great that the seatbelt doesn’t really matter. But probability-wise, it’s the smart thing to do. You’re reducing your risk of serious injury substantially.

Sure. And there’s always those people who will tell you that. I was in a car wreck where the seatbelt literally did harm. But if you think about it the other way, it also kept them alive. So, it depends on what you’re measuring. If you’re looking at it from, ‘Did it do me harm?’ Well yes, it did. ‘Did it also keep me from dying?’ Yes it did. I think a lot of companies have lost that necessary perspective. I talk to a lot of companies that say, “We’re going to go scan everything.” And I ask: “Does that even make sense? Do you even want to do that? Do you have enough money to do that? Does that really make sense to you?”

So how should companies whittle it down?

There’s one metric called the Weighted Risk Trend (WRT). It basically tells you if you are addressing the right risks. Are you going after the things that matter? Because if you start your program, you start measuring things like how many vulnerabilities you find and how many vulnerabilities you fix. They found 10,000 vulnerabilities. They fixed 9,000 of them. That sounds great, doesn’t it? What if those 9,000 were all old, and the 1,000 that were left were the 1,000 that were critical?

Because the 1,000 that were left are high-risk and existing on business crucial machines?

Exactly. And the question is, how do you know the difference? And many of the folks out there just don’t. So the WRT helps you understand, yes, there is a difference between bad and really bad. A lot of the customers that we work with tell us they’ve had somebody else come in and help them first. They ended up with a process that looked like this: They’d scan applications and throw reports back at the developers demanding them to fix the errors. That’s not very effective and that certainly doesn’t solve the problem.

What is a more effective way to solve the problem?

Helping to improve the development workflow. Instilling the ability to correct design mistakes early, and spot and fix coding defects early in the process. To that end, you start having conversations with people in quality assurance and development teams. You decide to test code in various stages. After the tests you ask the various teams if they tested for security and quality defects. “Yes, we did,” They’ll say. They’ll say they did a great test. They’ll say they tested all the functional requirements. That they tested all the application use cases. Then the problems come back. The developers still keep making the same mistakes. That’s a failure in workflow. What organizations need to focus on is improving their development flow, so that they’re not just fixing vulnerabilities as they arise — but training and giving developers the tools and support they need to develop better software.

CSO’s Daily Dashboard gives you a one-stop view of latest business threats. We created it for you! Bookmark it! Use it!

How do you recommend organizations measure how well they are doing at improving secure development?

It’s what I call the SQR, the Security to Quality defect Ratio. It measures how many security defects you are generating as a function of quality defects as a whole. Are you generating a disproportionately large amount of security defects in your quality cycle? So, if you have 10 performance defects for an app, 35 functional defects, and 350 critical security defects, well, you’re generating a disproportionately large number of security defects. Clearly, your quality program is failing on security.

How does that information help to improve your secure development program?

Now, that doesn’t sound like what I would call an ultimately mature Key Performance Indicator. However, think of what just happened in what I just said. You can now have a conversation about how much security is impacting the rest of one’s software quality. How many companies can tell you that today? I’m guessing probably somewhere under 5 percent. Based on the conferences I’ve been to, based on the customers I’ve visited, very few organizations are thinking about security in the context of software quality. Those who are? Well, they are having tremendous successes.