The Metrics Quest

We have a chief financial officer who’s always been a nut on quantitative measures. But he’s recently decided to make a metrics march on all his direct reports—and that includes me. So every department in the company has engaged in a great exercise identifying the metrics appropriate to their business processes. And since all the service functions (including corporate security) report to him, I determined that compliance is the better part of valor.

I first found this metrics mania somewhat vexing. Historically speaking, I am the kind of person who has proudly sported that bumper sticker proclaiming, “What do you mean I’m overdrawn, I’ve still got checks left!”

I decided early on to go to the source, the CFO and chief metrics officer himself. What might he be looking for in this program? I was aware that there was a risk involved here; I didn’t want to appear like I was carrying a dunce hat in hand. But caution has never been my strong suit, so I got right to the point in our one-on-one meeting in his office.

Fateful Meeting

Me: “I know you have an endgame in mind with your measures request from the services group. In corporate security, we generate volumes of data on a daily basis. It would be helpful if we could kick some ideas around on how to meet your goal.”

CFO (smiling): “Sure. The heart of it is that if a business process cannot be measured in one way or another, we likely ought to cast it off as wasted effort. As a business, we live or die every day on a host of measures, all of which indicate our health to shareholders, the capital markets and any manager worth his or her paycheck. I know you’re thinking about replacing our global access control system. It’s a significant investment. You know I need to see the return on it. What’s the benefit? When is the payback? What’s it going to give us that the one we’ve got doesn’t? This is the simple stuff. Digging into the essence of what we get for our global spend on security programs is far more difficult.”

(He pauses, looks at me. I know this is a test. But I’m not done asking questions.)

Me: “You’ve highlighted the fact that we’ve always been seen as a cost center. But how do you see security as being a fundamental part of our financial success?”

CFO (excited, stands up): “That is absolutely the right question! But think about it for a moment. You are in the best position to answer it. And, you need to know that I don—t see security as a cost center. You are a performance center. You are focused on helping our company succeed in an increasingly risky world.

“Every business process involves a variety of steps, actions or transactions that can be measured. In my office, we track dozens of financial performance metrics that provide grist for boardroom debate. While a few years ago I might have wondered aloud about the value of a comprehensive security program in a global business, no more. The risks are all over the map, but I leave it to you to develop the metrics you believe tell the story. Come back with a focused approach and we—ll go from there.”

I leave the CFO’s office thinking, I couldn’t have asked for a more supportive charter than that, could I?

Metrics Cover More Than IT Systems

The quest for the right metrics has been illuminating for a couple of reasons.

First, I knew we generated a variety of volume-related statistics, for our internal performance assessments and to support our insurance and risk management requirements. But I have to admit that we hadn’t considered a more proactive use of this or other data—until it was made clear that performance metrics were going to be a much bigger deal.

Second, after some serious Web surfing and literature review, it became clear that the bibliography for “security metrics” was limited to say the least. Limited to information security, that is.

Two examples. The Robert Frances Group states (in a report posted on CSO’s website, www.csoonline.com) that “collecting and reporting security metrics is an integral part of an enterprise security strategy. IT executives should examine their metrics collection practices to ensure that the metrics collected are useful and understandable, and cover all necessary security aspects.” I thought “enterprise security strategy” covered the whole corporation? Why talk only about “IT executives”? A Security Metrics Consortium announced last February and founded by some leading CSOs and CISOs said it “was established to define such real-world metrics, giving CISOs and CSOs the ammunition to adequately protect digital assets of their organizations.” Digital assets? Do you mind if I list people assets and maybe reputation before all the zeros and ones? It’s a damn shame that professionals with their heads screwed on straight failed to include the whole landscape of security metrics in their leadership model from the get-go.

Our research on the Web confirmed that the IT security consultants and centers of expertise were the prime movers in security metrics. This highlights what I can only conclude is a general absence of interdisciplinary debate, benchmarking, project coordination and collaboration among security professionals. (In checking with some colleagues, I learned that metrics are a centerpiece of the reengineering consultants— pitch—but my budget wouldn’t pay for a day of their time.)

The dominance of the digerati is disappointing, but I have to applaud the fact that a core element of our security profession sees the wisdom of building a security metrics database. The digital side clearly has the machines to log and track metrics like intrusion attempts, access attempts to blocked Internet sites, virus alerts versus infections, and scanning tools that can detect a variety of vulnerabilities and prepare slick reports. But I learned early that an enterprisewide security metrics and measurements program is significantly broader and deeper than this.

Data on Hand, Awaiting Analysis

My group wasn’t totally in the dark on metrics tracking. We have always watched trends on losses, exposures, shrinkage, recovery, safety incidents, investigative caseloads (how many and how long cases are ongoing), various metrics to prioritize investigations, response times, false alarms, outages, and on and on. We have a global incident reporting system that probably houses terabytes of data. We just hadn’t considered how this information could be organized to make the business safer, more efficient, less costly, better managed and more accountable.

We were sitting on reams of historical data that failed to provide real information. We fed the databases hourly without fail. My investigative team conducts hundreds of cases annually, has good recoveries, makes successful referrals for prosecution and writes excellent reports. When we reviewed several of these, it was amazing what information we had failed to pass on to business units on process deficiencies, managerial failures, recommendations on vulnerability fixes. The take-aways from real incidents provide metrics that can make the company more profitable, measures that can make managers more effective and accountable.

All business continuity incidents now undergo a lessons-learned analysis, a routine that has identified significant flaws in our emergency notification lists, contingency plans and offsite recovery strategies.

Thinking more strategically about our metrics initiative also meant considering the multiple constituencies who would receive our reports. There are several stakeholders in this game. I knew I could work with the CFO, but to influence the business as a whole we had to identify others who would benefit from the knowledge we could give them.

HR and legal counsel were early partners. We could demonstrate to HR, for example, that our background investigations showed that certain headhunters were coaching applicants to inflate past salaries and embellish experience. By analyzing our internal investigations, we showed that first-line managers and supervisors would benefit from better training programs and beefed-up business conduct policies that incorporate strong ethics—the kind of information that can keep us off The Wall Street Journal—s front page. Similar results were achieved by drilling down on workplace violence cases.

Another result: I now meet regularly with our general auditor. Organizing our incident results has identified new items to examine in the next audit cycle. We—re more aggressive reporting to the board of directors as a result of Sarbanes-Oxley. Lessons LearnedSome lessons learned from this process:

Engage your internal business unit clients in identifying one or two metrics vital to their success. Consider: loss reduction (be specific), cost reductions, shorter cycle times, use of technology versus use of people, elimination of vulnerabilities that impact uptime, reliability and so on.

Risk analysis is a must. If you aren—t doing risk analyses I assume you are looking for a job. These projects offer a potential wealth of metrics and bolster your recommendations to corporate leaders.

Identify incident trends important to key senior managers. Track changes monthly or quarterly. Focus on what’s important in your business. Consider: safety violations, workplace violence, public safety, emergency medical technician response times, issues that invite regulatory sanctions, losses as a percentage of sales, numbers of employees who are subjects of business-conduct investigations.

Develop a few value indicators that you can track with a high degree of reliability. Candidates: security’s cost per employee as a percentage of sales or revenue, the property protection cost per square foot of occupied space, case cost versus recovery, case cost over time. And do some service cost benchmarking with your peers. These metrics tend to be more comparative in nature so make sure you are comparing apples to apples.

Set up a security council to develop metrics goals if security functions are spread among various departments.

Develop a couple of confidence indicators, such as annual customer satisfaction surveys posted on your corporate intranet. Or track business process improvement recommendations, made in incident postmortems to see which are accepted and which are rejected.

Build your annual business plan around two or three “reach objectives” that have at their heart a specific measurement like “in the next fiscal year, reduce background investigation cycle time by 15 percent and case cost by 5 percent.”

Lastly, keep it simple and check your numbers. Oh, and by the way, it—s all going very well with the CFO. I’ve even learned to balance my checkbook.