Metrics for Corporate and Physical Security Programs

Metrics are measures that matter, providing evidence of performance both to experts and to interested observers.

That’s why CSOs are hungry for them. It’s not good enough to maintain a quiet, reliable security service until something goes wrong. Security executives want to understand how their operations are working and how they can improve. CEOs want to know how the security function is faring by looking at

the department’s data. And metrics can provide the hard numbers and context on the performance of the security function, proving that nothing happening was the direct result of an effective security management program.

Key metrics vary by CSO, organization and industry. What’s important to energy provider Georgia Power (federal regulation compliance, for example) may not be important to coffee purveyor Starbucks (armed robbery statistics, for example). “Metrics resist uniformity,” says Dennis Treece, director of security for the Massachusetts Port Authority. “What works here may or may not work elsewhere.”

Moreover, CSOs say that metrics don’t always have to be straight-up numbers. Impromptu conversations with key executives can sometimes have just as much punch as a glitzy, chart-and-pie-graph show in the boardroom. “Clearly, statistics on their own don’t make a very good read,” says John Hedley, head of group security for food maker Nestlé. “You have to interpret them and put them into context.”

Here is the story of four security executives in different industries who give a rare peek into the physical security metrics that are important to them, their CEOs and their organizations. Taken together, these data points and measurements help them keep a firm grip on the most important metric of all: How much confidence the rest of the organization has in the security department.Starbucks Tracks Everything That MovesTo Francis D’Addario, the connection between security metrics and how effective he is as CSO of Starbucks is simple: His mission to protect people, secure assets and contribute savings year over year is validated with key performance indicators.

Whether D’Addario, vice president of partner and asset protection at the $5.3 billion coffee and food retailer, is talking about physical assets (stores and equipment), liquid assets (cash and coffee) or human assets (employees and customers), using metrics is how he judges the success of his security group.

First and foremost on the priority list, D’Addario says, is the safety of people. The frequency of armed robberies at retail outlets, for example, is an important metric at Starbucks and within the retail industry. He says that since 1996, when there were 46 incidents per thousand Starbucks stores, there has been a steady decrease to a best-in-class 11 per thousand in 2004. D’Addario says Starbucks’ numbers compare favorably to historic trends at similar outlets, such as quick-service restaurants (which have averaged 45 armed robberies per thousand) and convenience stores (125 per thousand). He uses metrics from uniform crime reports and industry associations.

D’Addario says the decline in robberies at Starbucks has resulted from implementing better awareness campaigns to help employees anticipate problems. Technologies, including smart safes and an interactive system that confirms security events, also have played a role.

Other metrics D’Addario relies on include tracking the frequency and outcomes of background identity checks, employee access control compliance (which is measured by spot audits and credentials checks), and cash or asset protocol performance (including sales, deposit preparation and banking). D’Addario says those are continuously audited, and exceptions are investigated routinely. “Cash loss is monitored as a percent to sales on every business unit’s P&L,” he adds.

D’Addario says that some measures he takes for security are also valuable to Starbucks’ quality assurance team. For example, tracking how well the company maintains the integrity of its food containers remains a critical interest for both his security group and quality assurance. Container integrity is the reasonable assurance that the contents shippedvia overseas and truck routesare those that were ordered. The company performs auditable inspections on these processes, including checking the integrity of container seals, he says.

Because Starbucks is global, method-ologies for tracking these processes vary by region, depending on the infrastructure and technology available. But the measures are an essential component of quality assurance, D’Addario says.

Key performance indicators are tracked by period, quarter, year-over-year and five years running, he adds. “That enables cost and benefit impact assessments, risk-gap closure analysis as well as return on funds spent,” he says.

The trend analysis that D’Addario documents allows him to test new security technologies and protocols against the trends to decipher if they are contributing to sales or net profitability.

Working in the retail industry, D’Addario also benchmarks his cash loss as a percentage of sales as well as inventory shrinkage numbers with reputable industry group figures. Those kinds of numbers (which he declined to share for publication) allow D’Addario to present security performance indicators to his bosses.

“Thoughtful prevention design with forecastable results for performance improvement are viewed as investment opportunities,” he says. As an example, he says that a number of international markets adopted exception-based reporting after witnessing its performance for top-line and bottom-line contributions in the United States. D’Addario reports that the protocol has since delivered the same performance in the international markets.

The key to all of that, D’Addario says, is that those forecastable results “are baked into the operational budget process with return expectations.” While that puts your security department on the hook for demonstrable results, it also can make the CSO look brilliant in the boardroom when he delivers.Nestlé Metrics Emphasize Prevention and Protection When there is civil war where your people are working, one physical security metric rises above all others: Keeping all of your employees alive.

For John Hedley, head of group security for Nestlé in Vevy, Switzerland, this scenario played out in November 2004 at Nestlé’s operations on the Ivory Coast. The West African nation has experienced constant turmoil between the government and rebel forces for the past three years. Hedley’s security staff, led by a regional security manager based in Abidjan, the commercial capital, set in motion an evacuation plan for the international Nestlé employees when it was clear that the violence was escalating to a dangerous level. The Ivory Coast produces 40 percent of the world’s cocoa, and Nestlé is one of the biggest purchasers. The evacuation of Nestlé’s expatriate staff was accomplished “with a minimum of hardship,” Hedley says. “While such an unplanned departure is distressing for all, at least we were able to set in motion some pre-evacuation plans.” Hedley’s group had reviewed those plans just three weeks before the evacuation happened.

For a global company such as Nestlé, with 115 production facilities in 86 countries, Hedley says operations such as the Ivory Coast evacuation are a necessary and expensive undertaking. Metrics enter afterward, in judging how well the operation went, what went into the preparation involved and the resultssuch as whether there were injuries or deaths.

“We have not done a cost-benefit analysis of how much money we have saved because of the security plan in place,” Hedley says, adding he was not sure of the evacuation’s cost. “We had more important things on our mind,” he says. “Having a plan in place and revisiting it once a quarter or year may be the most important metric of all.

“However, the costs can be reduced by effective contingency planningthe emotional cost for the staff concerned as well as the financial cost,” he adds. “Getting everyone out safe and sound means that there are no staff replacement issues. Keeping the factories and other buildings properly protected ensures continuity or early restart of production. These benefits could be measured if required.”

Hedley says he can’t apply blanket security and preparedness metrics around the world. “The ability to equate performance in one country, in one region, with another is difficult,” he says. “For example, our security officers in New Guinea are armed (but with bows and arrows), whereas in most places they are unarmed.”

Even with those impediments, Hedley does employ physical security measurements wherever he can. The areas most important to him are Nestlé employees, distributors and consumers; company property; and the strength of Nestlé’s reputation and brand.

Hedley says he focuses much of his attention on Nestlé’s brand and reputation among consumers. “We have a broad brand protection strategy, in which we work in close collaboration with the intellectual property department,” he says. “There’s a very strong argument that brand and reputation are worth more than physical assets.” Hedley points to the difference in measuring hard physical assets versus intellectual property and brand assets. “You can measure the number of burglaries you suffer and the amount of shrinkage,” he says. But in the order of priorities for his group, he looks to condensed milk as an example. “Stolen boxes of condensed milk can be replaced,” he says. “But if someone keeps them past the ‘sell by’ date, and then someone consumes it and gets an upset stomach, it’s not so much the actual value of condensed milk but the effect that the inappropriate distribution and handling of such products can cause to people.” And consumers’ upset stomachs tend to give him an uncomfortable feeling as well.

The bottom line is also important to Hedley and his bosses. “We [in security] are judged by our overall contribution to the profitability to the group,” he says. As an example, Hedley tells of how he grapples with trying to plan for the unforeseen. “Having the ability to reduce the number of events that are unforeseen is a very valuable metric,” he says. When he is able to do this, it grabs the attention of senior management. “If you can tell a story that says, We were able to preempt a problem that was going to affect us, and, Oh by the way, had we not done this, this would have been the costthat is a very good story to tell.”

CSOs can estimate the damage that was not predicted or planned for by comparing to previous events or ones that hit other companies, Hedley says. You can say, If we hadn’t taken the action we did, then the probability effect would have been X. “The downside, however, is that you can’t say, This is the money we would have saved, and go put it back in the bank account,” he says.Utility Uses Government Rules to Build Metrics Margaret Levine, corporate security manager at Georgia Power, has found ways to convert the necessary burden of regulation into a bounty of physical security data for the electric utility.

Levine must demonstrate that Georgia Power, the largest subsidiary of Southern, the $11.3 billion regional utility based in Atlanta, complies with federal regulations. Her security group does that by completing security audits to make sure that the protected areas at plants and substations are indeed protected.

“We have reports documenting that the people who have access to those areas have legitimate reasons to be there,” Levine says.

Tracking results of these and other reports yields a measure that allows Georgia Power to compare its performance to itself in past years. It’s a conscious management decision to turn the “play by the rules” portion of the operation into a performance measure.

“You need to find a meaningful purpose other than just pushing paper,” she says. Security executives, she adds, can “take the next step and think, How can I use this report and statistics in a way to improve my security program or to better educate me about my customers’ business?”

A second metric for Levine comes from a combination of readiness reviews and penetration testing.

Readiness reviews are planned events and are a key component of Georgia Power’s business continuity program. The reviews assess whether employees and site security professionals at a particular facility understand that facility’s threat plans and know what to do when the threat level is raised or lowered. Readiness reviews also include interviews with local managers about facility security; an audit of procedures and documentation related to security requirements; an evaluation of the facility’s physical security program; and a review of its emergency action plan.

At the end of each review, Levine says, her office writes a report for the facility manager that highlights findings, best practices and recommendations.

For readiness reviews, Levine sends a team of security professionals unannounced to do security audits of all critical facilities and operations (though she declines to list what types of facilities those are).

In addition, penetration testing attempts to breach securityprocedurally, technologically or physicallyto determine whether the security program is functioning as it should, she says. “We may have someone try to walk through a facility without wearing a badge to see how far they can get before being challenged,” Levine says. “Or we may have someone see if they can talk their way around our delivery processing requirements.” Results ReportsResults are reported in two ways. First is what Levine calls the “objective, scenario, outcome”: Here’s what Georgia Power was testing (for example, the effectiveness of visitor management personnel); here’s how security tested it (use of outdated or fake identification credentials); and here’s what happened. “The results are reported by comparing the test outcome with the test objective, in addition to including a description of how the test was carried out,” Levine says.

Second are the lists for “did well” and “areas for improvement”: These are reported along behaviorally based criteria (for example, clarity of communications with “outsider” or whether incident notification procedures were followed) as well as results-based criteria (penetration foiled or speed in which penetration was detected).

After collecting results, Levine’s group tracks the physical and technical security measures at each location to ensure that they are functioning properly. Physical security measures include perimeter barriers, lighting, locking devices and key controls, and signage. Technical security measures include intrusion alarms, closed circuit television and other monitoring devices, access control and visitor management systems.

“We would want to make sure that the security folks onsite knew what to do in the event of raising the threat level or a breach of security,” Levine says, “and also have a good awareness of security protocol and who they could go to if a breach did occur.”Tracking TrendsIncident trends and loss trends are next on Georgia Power’s metrics list. Levine says that it’s critical to be able to demonstrate that a CSO’s security program is a significant mitigating factor in preventing increased incidents and losses. Levine can compare incidents by quarter, year-to-year and across multiple years. She can note the changes in the number and frequency of incidents by type of incident (for example, thefts, threats against employees or sabotage), by line of business (generation, transmission, distribution, staff services) or by location. She follows the same process for tracking losses; she says she tracks property and monetary losses. The key, she says, is if you’re not able to prevent losses, then “you can demonstrate an ability to quickly pinpoint where the weakness was and put in place the appropriate stopgap measures.”

Levine adds that metrics must be more than in-house security tools; they have to be relevant to the people she supportsbusiness executives, plant operators, substation engineers, customer service managers. She says her reports must contain information that is important to them, not just to security managers. Doing this, Levine says, “also enables us to educate them about things that are important from our perspective, and in that give-and-take process we’re able to validate the measures that we’re using.” Depending on the type of data and compliance requirements, Levine reports her metrics monthly, quarterly or yearly.

Levine considers two other factors when collecting data for metrics. The first is how Georgia Power compares to other utilities. And the second is data quality.

Levine says Georgia Power collaborates on metrics reviews with other security managers from within Southern’s 12 operating companies. (Besides Georgia Power, there are four electric utilities and companies in wholesale power, power generation management, natural gas, nuclear power and energy services. Southern also owns a wireless company and a fiber optics business.)

As for data quality, Levine says that it’s important to watch out for the equivalent of scorekeeping changes. She says Georgia Power recently transitioned from a 10-year-old case management system to a new system developed last year by Southern’s security managers. The case management system is a database that records all the details of incidents that are reported to corporate security. This includes an incident narrative and summary; victim, witness and reporting party names; losses; investigative activity; and case resolution.

Building the new system required a review of incident definitions so that a year-to-year comparison made sense, she says. For example, the old case management system had separate incident categories for burglary, larceny, fraud and robbery. But in the new case management system, all of those crimes are categorized as financial matters. “To make an apples-to-apples comparison between the old and the new, we have to select a specific subcategory (for example, larceny) in the new system,” Levine says. “Otherwise, the analysislarceny versus financial matterswould show that we’d had a crime wave at Georgia Power.” And that’s the last thing that Levine and her executives want to hear.