Lies, Damn Lies, Security Metrics, And Baseball

The legendary British Prime Minister Benjamin Disraeli is said to have noted that “There are lies, damn, lies, and statistics.” Much of the technology world is focused on statistic and metrics. You’ve often heard it said “If I can’t measure it, it doesn’t exist.” Known as the McNamara fallacy – named after the business tycoon turned Vietnam era Secretary of Defense – this famous idea failed miserably as a strategy. While it sounds good to the CEO’s ears, there is a corollary bubbling up below him that implicitly states that “If my boss wants to measure something that doesn’t exist, then I’ll invent it!”

This is especially true whenever leadership is disconnected with the field. As Big Data gets Big Buzz, the promises will become self fulfilling. David Hackworth, reportedly one of the most decorated soldiers in the Vietnam war, explores this premises in Sam Adams’ book “War of Numbers: An Intelligence Memoir” with an introduction by Hackworth. I met Hackworth in the late 1990s on the set of “Job Bob Briggs Drive In Theater” where I was the show’s Engineer in Charge. Hackworth didn’t look like a war hero. While his stature was Audie Murphy-ish, his legend was huge. I remember him as always smiling and joking – cheerful despite the way he had been treated by his superiors in the military. His mantra was how the Vietnam War was mismanaged by disengaged leaders who were more focused on counting dead bodies than they were on the strategic aspects of warfighting. As he clearly demonstrates in his books, this was a disastrous approach, as the field officers like Captains and Lieutenants were incentivized to invent dead enemy to increase the body count. I fear that this may happen in the InfoSec world if we become more focused on metrics than strategy and on providing glowing reports to our superiors than truth telling.

Warning: CISOs beware of metrics.

Too much focus on them will force your teams to manage to those metrics instead of telling you the truth related to your organization’s security posture. Hackworth was a truth teller and it cost him his career. He never rose above Colonel and he was eventually drummed out of his beloved military, but history has vindicated him. The eternal reputations of his superiors are tarnished. Generals such as Westmoreland and Abrahams, whose aggressiveness in WWII earned them accolades and rank, are now relegated to the waste bin of history.

Ultimately metrics must be balanced with your gut. This lesson was brought home to me on a recent flight. Evidently the gentleman sitting next to me was famous because everyone else coming down the aisle were pointing, staring and whispering to each other. Eventually I said “Sir, clearly you are famous and I apologize for not recognizing you.” He waved that off with a smile and a flick of his fingers. It turns out this gentleman manages a major league baseball team. Not being much of a baseball fan, he proceeded to educate me about the finer points of the game. During our conversation I mentioned the movie “Moneyball” which was in theaters at the time. He scoffed at the concept. “This game,” he said, “is all about the gut. You have to have an instinct for the game.” He said that while the whole Sabermetrics/Moneyball thing worked for a short period of time, it didn’t take the rest of the league long to figure it out. It was a short term solution that glossed over the lack of “gut.” These metrics were unsustainable once the opponents understood them and he dismissed the Oakland A’s as an also-ran once again.

I had a moderate interest in baseball on that day as I live in Dallas, and the Texas Rangers were in the 5^th game of the American League Championship Series that night. So I asked this baseball legend to tell me what would happen in that evening’s game. To demonstrate the importance of “gut” he predicted the Ranger’s destiny. “The Rangers will lose tonight,” he said, “because Justin Verlander is pitching. They’ll win the 6^th game and go on to the World Series where they will lose in seven.” That’s exactly what happened. He didn’t input any numbers into a spreadsheet to find this out – in fact he didn’t even have a computer with him. He just checked his gut. It knew the answer.

Somewhere in your organization is someone with “gut” – someone who can look at a problem and intuitively understand it at its deepest level and probably solve it. Don’t get so caught up in measuring things that you don’t do a “gut check” once in a while. Find the David Hackworths on your team and listen to them. They’ll tell you the truth. They won’t pull things out of the Ether so they can be “measured.” Yes, sycophants are more fun but they won’t keep your organizations from being hacked.

I say all of this because I will be in Austin this coming weekend for South by Southwest Interactive. I’m on a panel with Andrew Hay and Mark Seward called “Big Data Smackdown on Cybersecurity.” Don’t get me wrong. I’m all about Big Data. It will contain lots of valuable information that IT and Security pros can use. But I worry that our love affair with Big Data will blind us to the obvious, much the same way the body count emphasis blinded our military leaders and kept them from actually winning the Vietnam war.

At the end of the day my security executive friend, listen to your “gut.” It won’t let you down.

If you’re in Austin for SXSW this weekend make sure and come to our session.