TSA’s Risk-based Approach to Security

Part 1: Lift

Terminal A in Boston’s Logan International Airport is so new it still has that new terminal smell. It’s absurdly spacious with ceilings of significant altitude. Trash bins, benches and planters (currently holding poinsettias)pretty and also designed to bear up against bomb blastsrun the length of the sparsely adorned building, parallel to massive (blast-resistant) panes of glass that make up the terminal’s front wall and look out on a state police officer who tells lingering motorists to move along on this bright blue day just before Christmas. The terminal is subdued. A few passengers check in and watch as their checked bags are conveyed behind a curtain, out of sight. George Naccara, Logan’s federal security director, the Transportation Security Administration’s head man here, hustles past the counters and down an escalator. He’s going where the bags are going.

At about the same time, Rep. Ed Markey (D-Mass.) is hosting a press event in Terminal B, an older facility with low ceilings and narrow hallways. Markey is introducing the Leave All Blades Behind Act, legislation that would prevent TSA from removing some scissors and tools such as screwdrivers from its banned-items list, which was scheduled to happen two days hence. This change is a small but controversial part of Naccara’s broader, ambitious agenda to reinvent TSAat Logan anywayas a risk-based decision-making organization. By whatever small amount the banned-items list reduces risk, Naccara argues, it’s not nearly proportional to the resources it requires.

Every month, Logan’s TSA screeners confiscate 12,000 banned items from the traveling public. Pocket knives, knitting needles, scissors and the like make up the vast majority. Once in a while, a serious blade shows up, as does the occasional ice pick. Screeners have confiscated a can with the words “Time Bomb” stamped on the side. It was perfume. Every so often they’ll seize a carburetor. Once, a research doctor tried to stow seven human heads in the overhead bin.

“But the point is, 11,995 of those items pose no risk at all,” Naccara says. “They’re people going about their business who say, ‘I forgot that was in there.'”

A more effective security operation, Naccara argues, will not waste time looking for and confiscating scissors and knitting needles. It will watch the behavior of the people who carry scissors and knitting needles; and it will use technology to look for more serious threats in a way that’s both less random and more comprehensive than current methods. Naccara’s insistence on ending the confiscation madness and reinventing TSA comes off as a kind of benevolent belligerence. He’s also received some support from his boss in Washington, D.C., Kip Hawley, who has similar ideas about managing risk.

But Naccara’s dilemma is that for years now his own agency, TSA, has very successfully marketed the need for and effectiveness of the banned-items list. Maybe the flying public have an object fixation, but TSA did the fixing. Travelers have come to accept “no sharp objects” as a fact of flying. Naccara, then, must replace one marketing message with another one potent enough to redefine the public’s perceptions of safety. That’s no small thing. People might not care to be safer if they don’t feel safer. And even if risk analysis says reducing the banned-items list is not risky, passengers won’t feel safer knowing sharp objects are allowed back on planes after hearing how important it was to ban those items in the first place. And many of those passengers vote.

Which explains Rep. Markey, over in Terminal B, calling TSA’s plan “a gift to terrorists this holiday season.” Markey has enlisted the support of congressmen from both sides of the aisle, as well as the Association of Flight Attendants and family members of 9/11 victims. Even Craig Coy, CEO of the Massachusetts Port Authority (Massport), who sees Naccara almost every week, was, at first, publicly critical of his plan.

Downstairs in Terminal A, a local TV political reporter and his cameraman stop Naccara. Care to comment on Markey’s position? Later, Naccara promises. The reporter and cameraman retreat, and Naccara presses on. Finally, through one more door, he has caught up with the bags inside of a 3.5-acre room that smells of jet fuel. In fact, it’s the country’s first in-line integrated baggage-screening system.

“Here it is,” Naccara says, as if he wishes the reporter were still here, because this room is, in fact, how he cares to comment.

When Naccara arrived as Logan’s security director in June 2002, the airport’s security was terrible; and it appeared, from the details that roiled up in 9/11’s wake, that Logan security had been terrible for a decade. Massport, the governing authority, was savaged, accused of cronyism in hiring security personnel. Federal Aviation Administration reports surfaced that said agents had easily slipped guns, inert hand grenades and simulated bombs past checkpoints. All told, there were 234 such violations at Logan in a decade, the fifth-highest total among major airports. FAA agents also managed to get onto Logan’s airfield 26 times. Once, a teenager scaled a perimeter fence, crossed two miles of restricted area and stowed away on an international flight.

Castigated by all this (and further motivated by a close-up look at shoe-bomber Richard Reid when his flight was diverted to Boston in December 2001), Logan seemed to repent and to zealously pursue forgiveness in the form of improved credibility. “The terrorists had about a 75 to 80 percent chance of succeeding hereat any airport, reallyand they did,” says Tom Kinton, airport director for Massport. “We know there’s no silver bullet, but we realized that what we had to do is flip those odds. Make it a 40/60 or 30/70 proposition. Terrorists, like any competitive force, won’t go anywhere where success is a 50/50 proposition or worse.”

Almost three years to the day after terrorists used it as a launching point, Logan was named the safest airport in America by Airport Security Report. A year after that, Kinton won the coveted Airport Director of the Year award from another trade publication. Perhaps more important than the awards, though, the airport had developed what many call “the Boston reputation” for security. Logan has become something no one could have predicted it would right after 9/11: a successful and creative security innovator and an incubator for new security technology.

Logan volunteers to test whatever new security technology it can. The Terminal A baggage room is one example, but the airport also tried similar technology out front at security checkpoints, under a program called Cobra (Carry-On Baggage Real Time Assessment). Logan tested and now uses the explosives trace portal (ETP), or “puffer machine,” at one checkpoint. The ETP shoots several bursts of air at a passenger’s body. The jets of air dust up microscopic particles, which are analyzed for traces of explosives.

Taking input from screeners, Logan reconfigured the screening process at the terminals’ security checkpoints and increased passenger throughput by 30 percent. Naccara will offer up how he did it to any airport that asks. Few seem to (more about that later). Logan is also the lone airport to mandate training employees and ticket agents in some basic behavioral profiling (think neighborhood watch rather than professional law enforcement), under a program called Logan Watch. The airport just started testing a new system that monitors the exit doors at gates and other restricted areas, looking for people going in through the out door and vice versa. TSA at Logan also wants to launch a broad networking project (wired and wireless) to link up security personnel and devices across the airport.

And there’s a general grant of permission to invent. Anthony Ventresca, one of several Logan veterans who came over to TSA from the airlines, was in the supermarket one day and noticed how several checkout lanes were earmarked for different numbers of itemsseven or fewer, 11 or fewer, and so on. It seemed to him awfully specific. Eventually, he learned that supermarkets use throughput analysis to configure their checkout lanes. Wouldn’t that work at security checkpoints in airport terminals, Ventresca wondered. Naccara says he told Ventresca what he tells any TSA person looking to try something out: “Go for it. I can’t really give you any money or people, but give it a shot.”

So Ventresca built a software program on top of a spreadsheet to collect data from the terminals’ security checkpoints. Because of it, TSA at Logan has shifted from guessing at passenger loads to predicting them with remarkable accuracy. TSA Ops Center guys in Boston can predict how many people will be coming through a terminal, what types of people (business travelers, school vacationers), and the amount and type of baggage they’ll have. Staffers can even predict, based on all this data, the number and type of security events to expect at any given terminal on any given day of the year.

It’s real risk analysis, the kind of thing Naccara loves. Ventresca says it’s an ad hoc tool in a constant state of upgrade; it doesn’t even have a name. But other airports have recently begun borrowing the software to see if they can do throughput analysis the way Logan does.

Logan has also implemented other, nontechnological innovations, such as injecting some degree of randomness into the security profile. This may sound counterintuitive, but making the profile more variablefor example, by occasionally adding canine units and semiautomatic weapons to patrols, or changing the screening process from time to timemakes the airport a less desirable target because there’s no predictable pattern to break.

But the most important nontechnical security that’s been added is behavioral profiling. After 9/11, Massport hired Rafi Ron, the former security director of Jerusalem’s Ben Gurion airport, as a consultant to assess Logan’s security and suggest improvements. A cornerstone of Ron’s advice was behavioral profiling, which uses techniques long employed by the Israelis to discern potential malevolence revealed through physical tells (stiff torsos, a rapidly quivering adam’s apple or clenched fists, among others). The program teaches screeners how to detect these tells and respond to them with techniques like “walk and talks.” (For Katherine Walsh’s interview with Ron about his methods, see “Suspicious Minds,” www.csoonline.com/020106.) Law enforcement and TSA personnel trained in the programs say that once you’ve learned behavioral profiling, the difference between an average nervous flyer and a suspicious one is stark. It’s as if the suspicious person were dyed purple.

The Massachusetts State Police have been using a program based on Ron’s at Logan for several years. “I watch those guys do this,” Naccara says. “They impress the hell out of me.” So do the techniques, and from early on Naccara wanted TSA to use behavioral profiling. Massport and the state police trained TSA people, and Naccara linked the programs together, developing clear protocols for handing cases off between one agency and another.

For an airport, such artful cooperation is somewhat unique; at Logan it contributes to the Boston reputation. For now, TSA’s program is called the Screenings by Passenger Observation Technique, or SPOT for short (the acronym has changed no fewer than four times, and at least three of them are currently in circulation). Whatever it’s eventually called, SPOT is the sun of Naccara’s solar system, around which all other risk-based security techniques revolve.

At first, he says, there was reluctance in Washington to move so quickly with SPOT. “We were pushing too hard and too fast for them,” he says. “But we didn’t back off, because we’d seen it work here. We knew it was the right thing to do.” Naccara’s aggressiveness could have backfired, but instead it aligned with the appointment of Hawley as TSA’s new director. Hawley wanted fresh ideas on aviation security, and he embraced SPOT and decided to name Naccara the program’s national director, meaning that Naccara would oversee the rollout of SPOT to the country’s 40 highest-risk airports.

At about the same time Naccara was appointed director of SPOT, Hawley was re-centralizing TSA in Washingtona decision that seemed dubious to many TSA officials scoring success after success in Boston. “Every airport is different,” Ventresca says, walking along a blast wall that backs one of the runways. “Physically they’re different. In the way the port authority works. In the relationships with law enforcement.” At any rate, re-centralization meant all directors worked out of Washington, so Naccara would have to move to direct SPOT.

But he told Hawley he would lead the SPOT rollout only if he could remain at Logan. Hawley, fully validating the Boston reputation, relented.

Part 2: Drag

In leveraging the Boston reputation, TSA at Logan is part research lab, part startup venture, andowing to a righteous belief in its methodspart religion. That’s right, religion. For Naccara’s TSA acts in some ways like the Church of the Managed Risk, determined to atone for past sins and eager to bring its gospel to other airports.

Given that Logan’s religion was born out of 9/11, one might expect that it would be easy enough to propagate. But it’s not. Now, nearly five years since the attacks, it’s becoming clear that other airports, politicians and the public may not have the energy or desire to adopt Logan’s approach to aviation security, even if it leads to TSA’s reinvention as a risk-based decision-making operation.

The best example of Logan’s inability to spread the gospel comes from one of Massport’s and TSA’s proudestand simplestinnovations, what is called “the 0830.” Every day since Sept. 12, 2001, major stakeholders at the airport have met at 8:30 a.m. for a security briefing led by Kinton.

On a typical weekday, as many as 75 people will attend, representing Massport, TSA, numerous carriers, state police, the FBI, the Centers for Disease Control and Prevention, Customs, air marshals and others. The meeting could last anywhere from 10 to 30 minutes. It’s a chance to review news and share intel, but it’s also a chance to communicate. Naccara says mini-meetings break out before and after the 0830, chats that he says have produced some of his best ideas. Russ Webster, TSA’s number two at Logan, says the meetings have helped because “we’re not trading business cards during a crisis. Everyone knows everyone, and we can get on with fixing it.”

Despite the required commingling of dozens of stakeholders, airports are often territorial and disconnected, which makes these meetings something of a security breakthrough. Naccara has offered the 0830 idea to TSA leaders at other airports, and to other security directors. Shaking his head, he says, “I can’t get one other to do it.”

Naccara, Kinton, Ventresca and others were happy to speculate on why and how Boston earned its reputation. There’s the visceral effect of 9/11; the local technology and defense companies, and major research universities; some credit Kinton and Naccara as well for their leadership skills and deliberately apolitical mode of operation.

But no one cared to speculate as to why Logan hasn’t been able to spread its practices to other airportseven though some of the ideas, like the 0830 meeting, are remarkably easy to implement at low cost. Kinton’s reaction to the question was typical: “I don’t know. I don’t want to make a judgment. I’m sure [other airports] take security seriously. If they haven’t adopted these practices, it’s not for me to judge.”

In some ways it makes sense to be pessimistic about Logan’s prospects for preaching the risk-based religion effectively. It’s not yet clear that Logan itself will succeed in the transformation. Whatever success Naccara has enjoyed is based on working within a fragile ecosystem comprising scads of interdependent stakeholders, agendas and jurisdictions (one security incident at Logan could involve 20 agencies).

For example, technology is now being tested to prevent people from using exits to enter secure areas and entrances to exit them. The question of jurisdiction over these doors is complicated, involving Massport, TSA, the airlines and the state police. But, says Ventresca, “Massport is shelling out the dough for the technology, [and] TSA’s going to test it.”

The fact that the various stakeholders managed to overcome ego and turf issues enough to get the tech trial off the ground counts as a minor miracle to some. “It’s not like this at other airports,” says one Logan TSA staffer. “And it’s barely like this here.”

Logan’s TSA managers are quick with stories about how behind the shiny success story there exists a creaky scaffolding built from these sorts of tangled relationships and fudged jurisdictions. Sources recountedthough not for attributioncountless turf battles and other comic scenes where a security incident would lead to “eight or nine guys in suits, all from different agencies, arriving at the checkpoint at the same time to take credit.” One manager ranted for 10 minutes about TSA’s IT supplier having ridiculously restrictive controls that hamper the agency’s ability to be flexible. That IT supplier puts stickers with its logos on all its equipment. As a passive-aggressive commentary, an anonymous TSA staffer started pasting those stickers on TVs and elsewhere around the office.

The TSA Ops Center in Boston, where staffers not only manage Logan security but also take in most of the security intel from across the country (it’s also where Ventresca’s throughput analysis software runs), still uses a dial-up network connection. “I feel 10 years younger in here,” one TSA staffer in the office deadpans.

Besides slagging the IT supplier, TSA employees were heard to rip Congress and the media for ignorance about risk; the FBI for its turf battles (including an incident where FBI agents locked TSA staffers out of a TSA office because the agents were having what they said was a confidential meeting); its own technology research group for a lack of vision and purpose (“they’re nice people, but useless; we basically work around them”); and the agency’s headquarters in Washington for not pushing even harder for the Logan way, and thus slowing progress toward better aviation security.

Naccara is more diplomatic, saying that the cooperation that exists at Logan is exemplary, that the subjugation of egos and managing of red tape is a positive, and that his relationship with Washington is “good.” But those underneath him are blunter. “Dial-up!” the Ops Center staffer snorts. “Can you [flipping] believe it?”

Even if Logan manages to hold the partnerships together, Naccara’s vision for security faces other obstacles. Troubling media reports of poor judgment on the part of TSA screeners at security checkpoints surface regularlyfrom improper pat downs to unusually harsh detainments. (Discipline is a heavily regulated process; in Boston a barrel-chested ex-Marine named George Barris is in charge of it. In the Ops Center, he held up a stack of paper, about 80 pages thick, which he said concerned a single complaint against one screener.) Naccara believes most lapses are a function of staffing challenges. TSA screeners at Logan Airport view 2.1 million images a month. “It is a repetitive, thankless job where you’re asked to invade the personal space of people who are already nervous about flying,” he observes.

Perhaps because of this, TSA suffers high turnover, another factor working against the long-term success of the Logan experiment. An internal review found that many who leave TSA are staying in government jobs, but move to agencies where the work is less mind-numbing and the job appears to have a career track. Naccara’s vision of risk-based decision making may improve the turnover problem, because it aims in part to reduce the repetitive aspects of screening and introduce more variety. Naccara hopes the behavioral profiling job will give TSA some of the allure sought by those who would decamp to other agencies.

But behavioral profiling itself carries another set of challenges. As the program becomes more public, concerns about racial profiling have been raised. The American Civil Liberties Union sued and then settled with TSA in 2003 over the practice in its earliest stages of development (a case involving the arrest and detention of a doctor of Indian descent by federal air marshals in Philadelphia). And in late 2004, an ACLU lawyer who is black says he was detained by state police at Logan Airport for no good reason. This is likely how it will be with SPOT; if concerns about racial profiling derail it, then the gravity goes out of Naccara’s risk-based solar system.

TSA’s Webster says that behavioral profiling focuses on physical cues, not appearance. “In fact, if you’re profiling by race, you’re doing it wrong, and you will miss people who would do you ill will,” he says. But no one is perfectly objective, and opponents point to studies showing that people profile racially without even realizing it.

Naccara acknowledges all of the concerns but doesn’t waver from his belief in SPOT. To allay fears, he says, he’s hoping to get an endorsement from the ACLUas he rolls out SPOT across the country. But he recognizes and worries about the problem. “One major incident and all this work could be for nothing,” he says fretfully. Adds Ventresca, “We’ll have to get through a couple of incidents and accusations before [behavioral profiling] is accepted. But we believe in it. We know it works.”

So, let’s say Logan manages to keep myriad stakeholders together, and then manages to minimize publicly embarrassing incidents at checkpoints and improve its staffing problems, and then manages to keep the behavioral profiling program intact despite challenges along the way. Naccara’s vision of a reworked TSA still faces another challengemaybe its biggest of all: metrics.

One of the Coast Guard’s many jobs is fisheries enforcementmaking sure fisherman aren’t fishing in restricted waters. For decades, Congress gauged the success of fisheries enforcement by one metric above all others: boat boardings. Like meter maids giving out tickets, the more boats the Coast Guard boarded in or near restricted waters, the better the job it was doing. Then came GPS technology, and the Coast Guard didn’t need to board so many boatsofficers could see where boats were going from their own vessels. Boardings dropped significantly. Russ Webster, who like Naccara was a Coastie, remembers that when it came time to review the Coast Guard budget, the first question Congress asked was, “Hey, what happened to fisheries enforcement?”

Nothing had, of course, but the Coast Guard was a prisoner of its “metric of success,” as Webster calls it. When that metric declined sharply, enforcement was assumed to have declined sharply too. The same is about to happen to TSA with the sharp-objects and checkpoint-throughput metrics.

“It’s unfortunate, but our two metrics are how many knives did we take away and what’s the wait time at the checkpoint?” Naccara says. In its first years, TSA wasn’t afraid to boast about these metrics to Congress or the public. But moving to a risk-based approach could send both of them in the wrong direction. Fewer than 12,000 items a month will henceforth be confiscated at Logan, because fewer will be banned from planes. It’s unclear what will happen with wait times as more randomness and complexity are injected into the screening process. An increase in secondary screenings, focused on more serious threats, coupled with greater use of behavioral profiling, can lead to longer interviews and detainments, albeit for fewer passengers.

Naccara is trying to supplant those metrics with new onesfor example, arrests based on behavioral profiling. He also tries to highlight the savings from not shutting down a terminal because of a screwdriver found in a carry-on bag. But he admits that it’s an “extremely difficult issue,” and he worries that the risk concepts will be lost on the public.

Herein lies a classic security conundrumjust as relevant to one of the most serious security threats in the country as to someone buckling up a seat belt in a car: “How do you measure the effectiveness of deterrence?” Naccara asks.

While Rep. Markey introduces his Leave All Blades Behind legislation in Terminal B, Naccara gives a tour of Terminal A’s high-tech baggage system, the first of its kind.

The suitcases we passed at the check-in counterall of Terminal A’s checked bags, for that mattercome into this room and wend through a four-mile skein of conveyor belts. All of the belts are suspended from the ceiling, crossing paths, dipping over and under each other, diving down to a section along the floor and then rising back up; this must be what a hamster maze looks like to a hamster.

Each bag also passes through one of the seven explosives-detection machines set in the maze. The machines are MRIs for suitcases. If they find a worrisome density or shapeblocks of cheese and jars of peanut butter often set off alarms, as do books and, well, explosive devicesthen they mark the suspicious spot on the bag’s MRI image and send the bag along its way, with all the benign bags, until, near the end of the maze, the bags reach the Vertisorter.

The Vertisorter is what it sounds like: a conveyer belt that sorts bags vertically. It tilts down to send a bag to its plane; up to send it to an adjacent room where more TSA screeners receive both the bag and a 3-D color image of its innards, on which the suspicious spot is marked. The screener rotates, flips and zooms the image. He switches to a high-contrast black-and-white view, superior to color for seeing wires. He slices through the 3-D image looking for things hidden inside of other things, the same way a doctor would navigate an image of a lung looking for a tumor. The screener has about a minute to decide whether to send the bag to another person for physical inspection or to return it to the Vertisorter to be sent down to its plane.

Naccara boasts that Logan completed the project on time, in 2002. He says the system has saved the feds tens of millions of dollars, reduced the number of screeners needed at Logan from more than 1,200 to 850, and, most importantly, reduced the risk of crime and terrorism in the air significantly. Far more, he says, than taking away people’s Swiss army knives ever will. If he could, Naccara would have every suitcase at Logan snaking through rooms like this one. Then he’d add the suitcase MRI machines at gates to improve screening images there. He’d network the system so that images that trigger alarms could be shared instantly across the airportby TSA, Massport, the airlines, Customs, maybe even the CDC if the threat were potentially biological. All of this would increase security and free up his staff to focus on the core of his agendabehavioral profiling, SPOT.

In Terminal B, Markey is saying that TSA is “taking a gamble” by removing items from the banned list. In Terminal A, Naccara is saying that using TSA’s limited resources to confiscate nail clippers from grandmothers is the bigger gamble.