Los Angeles sees 100 million monthly cyber attacks

Los Angeles is famous for its warm weather and movie stars. But what may not be as well known is that it’s also one of the largest targets for cyber attacks in the world.

The city’s infrastructure in highways, water and power — and all the data behind it — supports 4 million residents in the nation’s second largest city. The city also collects data about Los Angeles International Airport as well as about the largest shipping port in the western hemisphere, where 43% of imported goods enter the U.S. The city government is even responsible for data related to elections, including yesterday’s national election.

It’s not difficult to see the enormity of the city’s security challenge, which includes protecting the personal data of city workers and residents.

“We receive a massive amount of automated cyber attacks every month, about 100 million,” said L.A. CIO Ted Ross in an interview. Even so, “we’ve made tremendous improvements in cyber security in the last two years under Mayor Eric Garcetti’s directive.”

Ross heads up the Information Technology Agency, one of 38 city departments. The agency has a $90 million annual budget and employs 450 IT workers.

The most obvious security-related improvement was construction of a $1.8 million Integrated Security Operations Center, which opened in late 2015 in a location Ross wouldn’t disclose. The ISOC consolidates threat intelligence from what previously was carried out in four different locations.

From that single ISOC location, working around-the-clock, eight cyber threat analysts on each shift monitor 240 million security-related daily logs from sensors and other endpoints located inside critical infrastructure. The ISOC consolidation has helped speed up threat response and coordination, according to IT executives.

“We get situational awareness from one single pane of glass,” said Timothy Lee, chief information security officer for L.A. The system uses artificial intelligence to recognize attacks and which critical system is under attack. “We identify the source, how critical it is and how to restore the system.”

Lee and Ross didn’t want to divulge all the cyber security tools the city uses, but Lee said L.A. does rely on Amazon Web Services’ GovCloud to share approaches with other governments and does business with FireEye, a company offering a wide array of cyber security products and services.

Last February, analysts at ISOC were able to identify 16 ransomware attacks in five city departments. “We identified the attacks across the departments, segmented them off, didn’t lose any data and didn’t pay any ransom,” Lee said. The city determined the ransomware attacks were zero-day events, Lee explained.

“There is constant coordination and information sharing performed by ISOC across the city departments and with the broader network of federal and other local governments,” Ross added. “This is only possible with ISOC and didn’t exist before. ISOC was directly involved in identifying the ransomware in February.”

L.A. shares its findings about attacks with the FBI, Homeland Security and the Secret Service. In all, that sharing reaches up to 2 million cyber professionals, Lee said.

“We’re not only trying to up our game around cyber defenses,” Ross said. “We’re in a position now where we’re truly unified with other governments in a cyber watch and cyber defense effort.”

Even though there are new flavors of cyber attacks every week, Ross said his biggest worry these days is still ransomware. “Ransomware is just so ubiquitous and the delivery system is so innocuous. Someone can attack a personal machine or shared drive. With 48,000 city employees we have a lot of ports, so we need to be that much better than the attackers.”

To combat ransomware, the agency bangs out the common drumbeat: “If you don’t know where you got a link or an email or a download, don’t click on it,” Ross said. “The average person doesn’t realize they could launch something very powerful by opening that email. Human beings are often the weakest link in the chain.”

Ross and Lee said they feel confident about the security behind their internet of things infrastructure, which is protected by frequent password updates and patches on endpoints. In a recent Distributed Denial of Service attack on DNS provider Dyn that made major web sites inaccessible, the Mirai botnet was deployed, perhaps by amateurs, to insecure IoT devices, including consumer devices like internet cameras.

“DDoS attacks are certainly a big concern,” Ross said. However, Lee said Los Angeles does deploy vulnerability management software and endpoint protection, including antivirus software — using both behavior-based and signature-based techniques.

“At least with a DDoS attack, it takes a [relatively] long time to develop and gives us some time to react,” Lee said. The city also relies on frequent penetration testing to check for vulnerabilities.

“Even though government gets a rap for being old fashioned and paper-driven, certainly large cities like L.A. have been very progressive,” Ross said. “We see how dramatically fast the cyber landscape is changing. We see how cities are stewards of assets that nobody else has.

“Government may have been able to get away with slow processes in the past, but the stakes are very high in these areas and, generally speaking, government has come around to taking things seriously,” Ross added.” The cyber security problem is an immense one, but security is like insurance. If an attack happens, you are a genius for preparing, but if you did nothing, you’d be responsible. We do not have a false sense of security.”