The Company that Did Everything Wrong, Part 2

It was 1 a.m. and we had been working on our client’s data breach for eight hours. Most of the team had been awake for 20-plus hours, and fatigue was starting to set in when Bob discovered something.

He realized that a piece of malware that was embedded in the phishing attack linked back to a website in Spain. “I did a little research and that site in Spain is a compromised host,” Bob told me. “These attackers are very clever. They mirrored the normal landing site with one that they set up that contains their exploits.”

“So instead of a visitor landing on the regular home page, they land on a compromised, hidden page?” Sam asked.

“Correct,” Bob said. “This has the hallmarks of an extremely sophisticated attack.”

Bob has a lot of credibility in my book, so when he talks about sophisticated attacks, I take notice.

“Why do you say sophisticated?” I asked.

“First, there’s the malware package. Looks like its polymorphic, changes its digital signature at every execution. So creating a digital hash won’t help us locate other infections,” Bob said. “Of course I won’t know for sure until we have a chance to send it to Dave for him to decompile, but I have a pretty good feeling that it is. Then there is the amount of research that had to go into crafting the phishing e-mail. It uses all the right buzz words, talks about a current project and even lists company employees who aren’t listed on the company’s website. Then there’s the amount of effort they took to hide their site. Take a look at the homepage for this site.” Bob turned his laptop around so everyone at the conference table could see the screen. “Here is the regular site: www.compromisedsite.com/index.html. But here is the link for the site that contains the exploit: www.compromisedsite.com/índex.html. See the difference?”

“They look the same to me,” said Sam.

“They did to me, too. For a long time. But here’s the difference: The uncompromised site has a regular Times New Roman letter i in the name: index.html. The compromised landing page uses the special character an accented i.”

It’s tough to notice, and of course that’s the point. Whoever compromised this site went to a lot of trouble to hide the fact from the Web owner, and even from fairly savvy computer users, Bob said.

“Did you have a chance to connect to the compromised site using one of our sand boxes?” I asked him.

“I did just a couple minutes ago. The results are very preliminary, but I think they confirm the fact that this is a well-planned attack. The site tries no less than five different browser attacks, and that’s if the user doesn’t click any of the links! I don’t know yet how many attacks the links launch. Also, the Spanish site seems like it might be a site that some employees of the client would need to access for normal business operations. That would explain why it was so important for the attackers to hide their presence on that server.”

Armed with that information, Sam and I met Victor, our Russian colleague, down in the server room. “Hey boss! Velcome to ze dungeon data center. Ha-ha! Guess vat I have found?” he asked.

“Tell me you have good news, Victor,” I said.

“Vell boss, ze system administrators turned off all ze logging. No more of doze damn alerts going off! But zey did not turn off everything. And I found vat zey did not! Look here.”

Victor showed me the log file index of the company’s antivirus software. “See how ze log files are small here, and here and here, too,” Victor points to the logs from Tuesday, Wednesday and Thursday of the previous week. “But look here. Friday ze log is four times bigger than for ze other days of ze veek. I tink somezing happened on Friday. And ven ve look here, ve see this IP address over and over and over and over. It’s an IP address from ze Russian Federation. Telephone company in St. Petersburg.”

He suggested we wait an hour and come back. St. Petersburg is at lunch time now but he said he’ll call them in an hour and see what information he can get.Back in the conference room Michael was sitting at the table talking on his BlackBerry. He was looking pretty ragged, his eyes bloodshot and a new coffee stain on his shirt.

“I’ll tell them. Right. Bye,” Michael said as he finished the phone call. “That was the CIO. He wants us to brief him before we make our presentation to the CEO and the board.”

“We’re making a presentation to the CEO and Board?”

“Oh yeah, sorry. Forgot to tell you. Tomorrow morning. Well, actually this morning at 7.”

Not much time to prepare anything since it was already 5:45 p.m.

A short time later, the systems admin who had been working with Victor burst into the conference room and said, “Hey man, your guy is down in the server room actin’ all crazy. He’s yellin’ some kinda gibberish on his cell and getting all red in the face. You better take a look at him before he busts a vein or something.”As if on queue, Victor threw open the near door of the conference room and came in. “I have information about ze group dat used dat server as demo platform for hacker convention,” he said. “Ze IP address that vaz in ze antivirus logs vaz registered to dis vireless telephone company. Dey told me dat dey rented this IP range to ze hacker convention. So ve have proof dat dis group attacked ze e-mail server earlier, because vie vould dey use a server dat was not already compromised? Dey don’t! Dis server vas compromised before!”

“Wow! Were you on the phone to Russia?” I asked. “Is that why the sys admin thinks you were yelling?”

“Russian people understand authority. Sometimes you have to make them believe dat you are ze authority. Dats vie I was little animated in ze phone call. Ha-ha-ha! Russian hackers compromised ze e-mail server some day earlier than last Friday. Ve don’t know ven ze original compromise happened, but ze computer vas used for demonstration at ze hacker convention Friday last veek,” Victor said.

At that point, Bob approached me. “Eamon, can we talk in private for a minute?”

Alone in Michael’s office, Bob picked up the phone and started dialing. He must have hit at least 30 keys before he put the headset to his ear.

“Tremendous Fury. Yes that’s right, Tremendous Fury. This is an insecure line. Yes,” Bob started spelling the name of the client using the NATO phonetic alphabet (Alpha, Bravo, Charlie, etc.). “I’ll look for an e-mail.” Bob hung up. “I have to wait for an e-mail but I don’t think the Russians are the culprits here, or at least not the cause for the call from the Air Force OSI. This thing just doesn’t have the feel of a Russian hack, at least as far as the phishing attack goes. I respect Victor and believe in what he found, but I don’t think that the phishing attack was Russian in origin. Maybe this box was compromised twice.”

“That’s a bombshell,” I said. “I’m going to wait before I hit that alarm bell. Maybe later on in the investigation we’ll have confirmation.”

Back in the conference room the entire team was there, including Michael, and we all had fresh coffee. Ten minutes until the meeting with the CIO.

Here’s what I wrote down:

• The company’s e-mail server was compromised on date unknown but earlier than last Friday;
• This server was used by a Russian hacking group last Friday;
• This group used the server for a hacking demonstration at a convention in St. Petersburg;
• On Monday of this week, an employee clicked a link in a phishing e-mail. This e-mail contained information about the company and a project that they are currently working on, so this attacker either had insider knowledge or did quite a bit of research about the company;
• All of the data from a different employee’s hard drive (not the one who clicked the link) was seen traversing an Internet link monitored by the Air Force OSI;
• A call from the OSI alerted the client to the fact that they had been compromised. The organization did not detect this themselves due to insufficient security controls;
• The malware from the phishing attack has an IP address hard coded into it which links to a compromised Web server in Spain;
• We know the attackers are on the network, probably even this minute, because they orchestrated the removal of 120GB of data from the above hard drive.

“Have I missed anything?” I asked.

“China. I just received an e-mail from a friend.” Bob said.

“That’s a bombshell,” I said. “OK, then. Let me change this.”

We are dealing with at least two attacks. The e-mail server was compromised by a Russian group and the phishing attack is Chinese.

How we would come to wish that the client had been compromised by only two groups. In the next few days, the team would make discoveries that risked national security and would culminate in the resignation of the CIO.

The author leads a Computer Incident Response team. He may be reached at the pseudonymous amonmadreen@gmail.com or eamonmadreen@hushmail.com.