The fallacy of expertise

What is a computer security “expert”?

I’ve always hated the term “expert,” especially when it’s being applied to me. Except on annual review day. Twenty-two years ago, when I first started my computer security career, I thought I knew the world because I knew how to hack Apple, DOS, Windows, and could disassemble a virus back into its assembly language origins.

Today, although I sometimes feel like I know a lot about a subject, most of the time I feel like I know only what fits on the end of a fingernail. I secretly live in fear that the person who thinks I’m an expert will ask me a question that will immediately reveal me for the fraud that I am. Such is the life of a computer professional. Stop studying for two months and become irrelevant.

But at the same time I’m constantly surprised by how many security experts, with years or even decades of experience, think they experts. Examples abound:

One of my recent favorite stories discussed how multiple security leaders were socially engineered into responding to and joining the social network of other bogus, but well known, security experts. The researchers created fake profiles that matched the well known security figures, and then invited other security people to join.

It’s interesting to note that a reporter was one of the people who didn’t fall for the scam. In this particular case, the security leaders may have joined because the security risk was lower than, say, running an executable sent to them in an e-mail. But I’ll bet that some subset of that same study population could have been tricked into running a bogus executable as well. “Hey, I’m getting ready to release a cool new security scanner that is light years ahead of the competition, and I was wondering if you could test?”

Another test of expertise is finding malware on someone’s system. All of us are probably asked by neighbors, family members, and friends to check their systems for malware. I frequently use the Sysinternals’ utility, Autoruns, to display all the auto-starting programs running on a particular system, and then pick out the suspicious programs.

(Yes, I’m ignoring rootkits and other subversive malware programs for the current discussion.)

But truth be told, with more than 20 years of anti-malware experience, I’m only guessing. I’m pretty sure I got what was and wasn’t malware right most of the time. I’m constantly surprised by how many executables, even from well known vendors, are not signed. And many program files, especially printer files, have crazy names that almost appear random. When messing with friends’ machines I usually err on the side of not removing software versus removing it on a hunch, mostly so I don’t have to visit again so soon because I’ve disabled some needed functionality they relied on every day.

And shouldn’t I, when faced with an untrusted computer that could be compromised, instruct them every time to format their hard drive and start all over? I would, but then I’d probably be the one that would be asked to do it.

Outbound host-based firewall rules are the same dilemma. Have you ever installed a new program just to be surprised by a prompt from your host-based firewall to open one or more listening ports? I recently installed a new office productivity application, and my firewall prompted me just after the install to open a port for some new program related to the install. Why is it asking? I don’t know. The vendor’s documentation didn’t mention it. So what do I do? I guess, and say yes. I’m usually afraid that if I don’t say yes I’ll break some much needed functionality. The truth is that most of the people that are asked to make outbound host-based firewall decisions are often guessing. Even the experts. And it’s also why I love host-based products that use artificial intelligence or community-based, centralized databases to make the decisions for end-users.

And I could go on and on. How do we make good trust decisions on what browser plug-ins to install when the world’s most popular add-ins are being security-updated twenty times a year? If I can’t trust the big vendors to do it perfectly, how can I trust anyone?

How about that new cool program or utility you just installed? There’s a good chance it has security holes in it too. The new security tool, that everyone installs, yeah, that probably has bugs and vulnerabilities in it, too.

We’ve long known that unless you wrote the executable yourself, you really don’t know what it is doing.

So, even with the world’s best computer security minds, we are doing a lot of guessing. And if it is that hard for us to make the right decision all the time, imagine how hard it is for the regular end-user, or your mom and dad? Sheez, they must just throw their hands up into the air, or let someone else do it.

Gotta go — my phone just rang and it’s my mother on the line.