SCADA System Security: Out of Control

Vitek Boden sought revenge. After he was turned down for a job with the Maroochy Shire Council in Queensland, Australia, the 48-year-old disgruntled techie unleashed his anger in early 2000 by hacking into the town's wastewater system at least 46 times. On two separate occasions, his electronic attacks (apparently he used a stolen laptop and a radio transmitter) led to pumping station failures that caused as much as 1 million liters of foul-smelling raw sewage to spill into parks, waterways and the grounds of a tourist resort. In the surrounding area on Australia's Sunshine Coast, creeks turned black.

Boden was a disgruntled ex-employee of Hunter Watertech, the company that had recently installed Maroochy's computerized sewage control system. Boden's attack became the first widely known example of someone maliciously breaking into a control system. But there have been other control system breaches, including, for example, a 1997 control tower shutdown at the Worcester (Mass.) Regional Airport and a Slammer-related disruption of the safety monitoring system at FirstEnergy's Davis-Besse nuclear plant in Ohio.

Electric utilities, oil and gas refineries, chemical factories and even food processing plants use control systems to digitize and automate tasks once handled by people: opening and closing valves in pipes and circuit breakers on the power grid, monitoring temperatures and pressures in reactors, and managing assembly line machinery. And because these systems are now connected to corporate networks, their vulnerabilities serve as an entrée into the guts of the nation's critical infrastructure. A malicious hacker or terrorist group could conceivably take down parts of the power grid, throwing the country into darkness; they could take out emergency telephone systems or disable the floodgates to a dam. Even scarier to terrorism experts is a digital intrusion combined with a physical attackthink 9/11, but magnify the chaos by adding an electronic knockout of regional or national communication and power systems. The intent is clearly present: Raids in Afghanistan in early 2002 discovered that al-Qaida operatives had scoured websites containing information on SCADA (supervisory control and data acquisition) networks in U.S. water systems and the electricity grid. Alarmism? Unfortunately, the people with detailed knowledge of control systems security say no. Control systems are designed for efficiency and reliabilitynot security. In fact, "It requires very little knowledge" to hack into a control system, says Juan Torres, program manager of the SCADA program at Sandia National Laboratories.

Experts worry that this issue is not getting enough attention from both government and the private sector, for a variety of reasons: technical ignorance, lack of funding and perhaps the absence of a major incident to date in the United States. Even with a concerted public-private effort, securing these systems will take years. Older, legacy controllers can't handle newer security technologies such as encryption; in fact, many don't even have enough horsepower to accept operating system updates or software patches. "How a control system works is different from an IT system, technologically," says Joe Weiss, the former technical manager of the Electric Power Research Institute's Enterprise Infrastructure Security program, now an executive consultant with Kema. "It's deterministic, cheap and old, with little in the way of computing resources. It's not in any way, shape or form designed to be a secure system." Compounding these technical challenges are a number of entrenched cultural and management obstacles. The people generally responsible for managing control systems are engineers who often have had little cybersecurity trainingor interest.

That's a lot of problems. And a recipe for potential disaster.Efficient, but Not SecureFor years, distributed control systems and SCADA systems (see "Talk to Your Plants," this page, for the difference) were designed with proprietary technology, and were physically and technologically isolated from the corporate networks that run standard IT applications.

Fatefully, the drive for efficiencies of cost and time led many companies to knock down the wall that traditionally separated those two types of networks. In the electric power industry, for example, deregulation led to more interconnectedness as executives sought more information from control systems to help make output and pricing decisions. Manufacturing executives wanted to pull up real-time information from, say, their assembly lines, to monitor how efficiently their factories were running. "As the networking evolution came through and local and wide area networks were installed, they were generally installed by IT. Operations, so as not to spend double the money, started using the corporate LANs and WANs for the control networks," Weiss says. Ultimately, this meant many control systems were connected to the Internet.

This linkage has profound security implications. Now control systems are exposedvia the Internet, intranets, remote dial-up and wireless capabilitiesto hacks, worms, viruses and other dangerous payloads. That exposure scares Jonathan Pollet, president of PlantData Technologies, who advises companies on control system security. "With each release of worms and viruses, there are more and more customers with downtime," he says. Pollet says the Sasser worm in spring 2004 took out several oil platforms in the Gulf of Mexico for two days. "They had firewalls, but worms crawled through commonly used ports like ports 80 and 139. If any type of connectivity is not turned off, a worm in a corporate network will crawl to control systems," he says. Another virus, SoBig, affected the dispatching and signaling systems of CSX Transportation, halting train service for four to six hours along the Northeast Corridor in August 2003.

Accentuating the connectivity problem is the growing move away from proprietary software toward standardized and off-the-shelf software and hardware. For example, Pollet notes that some SCADA software vendors use the same Microsoft connectivity tools found in products such as SQL Server and Exchange. "A worm written to take down a SQL server can take down a SCADA system that has nothing to do with the target server," says Pollet. The same vulnerabilities exist with other common technologies, from Unix to ActiveX.Worlds ApartGlance at the organizational chart of a typical large company and you'll see that cybersecurity falls under the purview of the CIO or, sometimes, the CISO. That makes sense; those execs are best qualified for the critical job of maintaining safe, secure and private IT networks. But who looks after the security of control systems? In most cases, Weiss says, the real answer is no one. The CISO knows IT security but nothing about the shop floor or the control systems. The VP of operations or manufacturing understands engineering and control systems but knows nothing aboutand has no budget allotted forcybersecurity.

John Maguire, senior security analyst at PJM, the world's largest electric grid operator (it covers a region that includes Baltimore, Chicago, Philadelphia, Pittsburgh and Washington, D.C.), sees firsthand the lack of operational security know-how. PJM's members include some 800 power sources, and Maguire serves as PJM's external security rep to those companies. He says security is a tough issue for PJM's membership. "We've pointed at the right documents and suggested best practices, but there's a fear of getting started or not knowing how to get started. It's a new set of responsibilities, and security isn't their core business. In [industries such as] banking and insurance, most of their business is about information. For our members, it's about producing electricity," says Maguire.

In fact, IT and operations groups are not just separate, they're often antagonistic toward each other, according to Weiss. The engineers responsible for control systems care about around-the-clock reliability. For that reason, all the workers with responsibility for, say, an electric power substation, might have the same user name and password to ensure no one forgets theirs if they're called into action at 2:00 a.m. to troubleshoot a system. If a CSO told them, wait, that's bad security, we need two-level authentication for anyone to gain access, it would just reinforce the perception among the field guys that IT doesn't "get" control systems, Weiss says.

Hampering a get-together between the two sides is the lack of an overall security policy in many companies. Torres says he is heartened by the fact that an awareness of the importance of cybersecurity policies is on the upswing, with urging from groups such as the North American Electric Reliability Council (NERC) in the power industry, for example. Torres believes more companies are putting such policies in place; however, he adds, "Whether they include the right things or not is another question." Can't Patch ThisIn a typical corporate IT network, hundreds (or thousands) of PCs, servers and other devices are packed to the gills with processing power and memory. CSOs can slap on the latest security technologies without much adverse effect on the network. On the other hand, many legacy control systems still run on Intel 8088, 286 and 386 processors. These processors are "adequate for the functions they have, but if you try to lay, say, encryption over them, they can't handle it. We're sitting with 30- to 40- year-old systems," says Ken Watts, director of infrastructure and defense systems at the Idaho National Engineering and Environmental Laboratory, which does process control research for the Department of Energy.

The insecurity of these systems is manifested in plenty of other ways. Control system communications used to be proprietary; that changed when those systems began getting hooked up to enterprise networks and the Web. "The SCADA commands are now going over TCP/IP and clear text, and are highly vulnerable," says Pollet. "There's no way for a [control device] to know the SCADA command is what it says it is; there's no authentication, no encryption. They're highly vulnerable to denial-of-service [attacks] and viruses." He adds that easily downloadable TCP/IP packet-sniffing toolssuch as Ethereal and Ettercapcan be used to read clear text. That would allow a hacker to read and capture user names, passwords and even commands.

SCADA systems also connect to a wide variety of other communications mediaincluding public telecom networks, wireless radio, and private microwave and fiber networks. In testimony before a House subcommittee looking into control system vulnerabilities in March, Gerald Freese, director of information security at American Electric Power, talked about the interdependencies between SCADA networks and the telecommunications functions that support them. "We have to keep in mind that telecommunications is vulnerable in its role as a transport medium. It is subject to attacks such as 'man in the middle,' where transmissions are intercepted and altered, redirected or destroyed. Also, many power plants and substations use modems [vulnerable to a number of intrusion exploits] to manage equipment such as breakers, relays and switches over telephone lines."

Because SCADA systems were designed for efficiency and ease of use, vendors enable their products to be accessed remotelythrough dial-up modems, wireless handhelds and the likeso that customers will have an easier time making fixes to systems, often with no authentication required. And companies often fail to install the same security measures on control systemssuch as firewalls and intrusion detection systemsthat they use to protect IT systems. But those technologies have their limitations as well, since they weren't designed with control systems in mind. For example, Weiss says a typical firewall filters Internet protocols such as TCP/IP but not control system protocols.

Patch management is another gnarly issue. The message from vendors sometimes seems to be: Patch at your own peril. That's because installing patches can interrupt the real-time functioning of the operating system, which could have bad consequences. "We had a control system supplier send out a warning letter to all its clients saying, Whatever you do, don't put in a patch for the Slammer worm. The patch will get you," says Weiss. Gary Sevounts, director of industry solutions at Symantec, notes that part of the problem is that it's difficult to test patches (or any other security technology) in an actual control system environment because of the requirement for 100 percent availability and predictable performance. "If there's a 10 to 15 percent hit on performance in a banking application, perhaps there's a delay, but the customer is probably OK," but the same 10 percent to 15 percent delay in a SCADA system can lead to a power blackout, Sevounts says.