Why assessing third parties for security risk is still an unsolved problem

A Forbes article is making the rounds right now about America’s most cyber-secure companies, and I can already see the cybersecurity outrage machine up in arms. Full confession: I haven’t yet read the article, but I’m about to. I’m writing this in two parts: before I read the article, and after I read the article.

Part I: What are the most cyber-secure companies?

If you ask me to list the most cyber-secure companies (what does that even mean?), here is my shortlist, in roughly the order I think of them:

Top tier: Google, Apple, Microsoft, Amazon

Second tier: Bank of America, Goldman Sachs, Fidelity, Capital One, Meta, LinkedIn, United Airlines, Akamai (full disclosure: I am the former CSO of Akamai), Cloudflare, Fastly

That list isn’t meant to be comprehensive. I spent five minutes thinking, “Who has a lot of data, or systemic control of systems at scale, and does a decent job of protecting it?”

Part II: The problem with third-party risk management

Okay, I see that United Airlines and Fidelity make the top 20 after reading the Forbes story. The other financial services firms I hypothesized mostly are in the top 100, but not one of the infrastructure companies makes it. Entertainingly, the authors of this list agree with me that United Airlines stands head and shoulders above the rest in its industry. Deneen DeFiore, United’s CISO, must be doing a great job. But that’s our only overlap. What’s going on here?

This list is made by SecurityScorecard, one of the flagship companies in the third-party risk management (TPRM) industry. The challenge that TPRM companies have is rather simple: Provide a mechanism for companies that do business with other companies to evaluate the risk that their vendors present to them, from a cybersecurity perspective. SecurityScorecard and its primary competitor, BitSight, use a similar methodology: Create a risk score (sort of like your credit score), evaluate companies, and score them. Sounds easy, right?

Nope. Imagine if the credit reporting agencies decided that they’d start evaluating large enterprises with the same credit scoring algorithm that they use for me, as an individual. Of course they’ll look awful! Think about the size of Google’s perimeter – all its publicly visible IP addresses – against Intel (which came in first on the Forbes list). One of these is predominantly a chip manufacturer, and I seriously hope they have a small external footprint. This tells us nothing about Intel’s cybersecurity practices, which I hope are heavily focused less on their website security (which contributes to their rating) and more on their product and manufacturing security (which don’t contribute to their rating). Google, on the other hand, looks like one of the slum lords of the internet. Their addressable IP space is one of the largest on the planet, so of course they’ll look bad from the outside at a cursory glance, especially if one of your measures is the size of someone’s attack surface.

The credit reporting agencies, for better or worse, have much more data than the TPRM scoring companies. They’re embedded throughout our financial system, collecting a lot of information that shouldn’t be publicly available. The TPRM scoring companies, on the other hand, are doing the equivalent of drive-by appraisals. They look at the outside of businesses on the internet and decide how reputable they are based on their external appearances. Of course, certain business types will look more secure than others.

The alternative to TPRM scoring is, sadly, the TPRM questionnaire industry, which is only marginally less unhelpful. This is an industry focused on shipping massive questionnaires to vendors, which take huge efforts to fill out. Dedicated teams then review the answers to search for any answer that looks like a “no” to follow up on (all mature vendors have by now figured out to never say “no” to any questions). There’s now an entire industry focused just on streamlining filling out these questionnaires.

The TPRM problem is yet to be solved. Companies have a real need to understand the actual risks they inherit from their vendors, including both intrinsic risk (risk the vendor brings to you) and usage risk (risk created by how you’re using the vendor). Unfortunately, neither the scoring space nor the questionnaire space are solving this problem.