Are you giving useful advice?

If you can’t measure it, it’s not actionable. If it’s not actionable, it’s not useful. And if it’s not useful, is it actually advice?

help support advice guidance signpost

Recently I had the pleasure of speaking at DevOpsDays Madison. It’s not a security event, but I took a security message to a developer crowd. I love speaking at non-security events because I feel like this is a place I can lend advice and lessons to people that tend not to hear them. Most security conferences focus on things well above the level of simple messages, so I consider these a real treat. The focus of my talk was treating security features as a Minimum Viable Product (MVP). The idea is we like to focus on building perfect solutions instead of building solutions that work good enough.

The presentation was a bit tongue in cheek as I did poke fun of the security industry, the crowd of course loved it. Most developers don’t see the security team in a positive light. They like to see someone speaking to them from their point of view which is the idea. If I’m seen as an adversary there’s no point in even trying to give out any advice. Instead I got to pretend I’m a developer for a day.

The one thing I didn’t exactly expect to come out of the presentation was the focus on actionable advice. I started with the OWASP Top 10 and gave the crowd 3 things they can do to make a difference in the security of their products. When I started writing this talk I knew I wanted to focus on the OWASP top ten, what I didn’t expect is that I would realize how little it’s changed over time. This tells me the Top 10 list isn’t giving out useful advice on how to avoid these problems.

Some would argue nobody is listening, but I like to think if the advice is useful, there will be an audience for it. Everyone knows security is important. Not everybody knows what they can do about it.

The idea wasn’t to solve all the world’s problem, the idea was to give them some real things to do. Nobody is going to remember 10 things, but they might remember 3. My 3 things were “use a framework”, “don’t roll your own auth”, and “use orchestration for managing deployment”. Those are ideas everyone understood and can remember.

The really interesting conversation happened when a few security professionals I know gave my deck a look. The immediate response was “You can’t turn the OWASP top ten into 3 things”. I of course asked why not, and the answers were pretty squishy. I maintain our current top ten advice isn’t working, turning it into a list of three won’t make things worse.

The realization I had was that security pros loves to give advice that has little value to our audience. The OWASP top ten list hasn’t really changed much in the last ten years. That tells me we’re not giving out the right advice. We’re giving out plenty of advice, that’s not the problem. It’s just not useful advice.

I always come back to hearing security pros telling everyone they should take security training to help avoid phishing links. The reality is this doesn’t work, if it worked it wouldn’t still be a huge problem. The issue isn’t people clicking links, the issue is our system aren’t properly protected with things like two-factor authentication.

So how do we fix this? With actionable advice of course! Many of us think we’re giving out actionable advice, but there’s a difference between advice that is correct and advice that is useful. Telling users not to click on links is correct, but not actually useful. Telling users to enable two factor auth is useful and can help prevent problems from phishing. It can be hard to understand which things are correct and which are useful. We don’t have a lot of experience in this space.

The best way is to look at the results. If you give out some advice that is supposed to prevent phishing, pay attention to phishing attacks. If there are less, you’re doing something right. If there’s no change or they increase, your advice doesn’t work. Advice you can’t measure is voodoo, don’t give out voodoo advice.

It’s important to measure what you’re doing. Humans are good at coming to wrong conclusions. The password advice from just a few years ago is a great example of this. We thought we were giving out important advice by having users pick crazy passwords then changing them every 30 days. The reality was that our advice ended up being harmful. Data helped us understand our mistake.

I would lump actionable advice and data in the same category. If you can’t measure it, it’s not actionable. If it’s not actionable, it’s not useful. And if it’s not useful, is it actually advice?

Copyright © 2017 IDG Communications, Inc.

22 cybersecurity myths organizations need to stop believing in 2022