Securing Facebook: With a little help from his 800 million friends

The eyes of the online world are on Joe Sullivan.

As the CSO of Facebook, Sullivan is without a doubt one of the most visible security chiefs in the business. He must mitigate myriad security and privacy risks not only for Facebook’s employees and corporate systems, but also for the social network’s 800 million members.

Sullivan, 44, joined Facebook in 2008. He moved to the private sector 10 years ago to focus on security, and before that he was a federal prosecutor for eight years. His legal background has come in handy of late, as Facebook has sued several people for misusing the service.

CSO contributor Lauren Gibbons Paul talked to Sullivan about the challenges of managing security on a rapidly evolving social network.

CSO: With all the publicity about privacy and security regarding Facebook, what do you regard as the biggest threats?

Joe Sullivan: I think the challenge with being at Facebook is that it’s always about trust. People need to feel secure when they use Facebook. When it first came along, people were not comfortable putting their photo and real name on the Internet. But that’s the way Facebook works—it’s your real name and real identity interacting with real people in your life. If [members] experience something that erodes their trust in that experience, they’re not going to come back.

[Also read The Robin Sage experiment: Fake profile fools security pros]

[So] we have to invest really heavily in security. That’s not just someone getting access to your account directly, but your experience of someone else’s account getting compromised. If your friend gets compromised, you feel it. It undermines your trust in your experience.

The network effort can be used for bad as well as good. That’s why we’ve invested heavily in security for a long time. No individual can be on top of all the different risks every day—it has to be orchestrated across a bunch of different groups.

I’m always concerned about the risk of a compromised account. There are high-profile individuals, companies and governments that use Facebook as a way to communicate. That means we need to make sure they’re comfortable coming on to Facebook and feel secure in using it. We’ve seen situations where high-profile accounts get compromised. That is guaranteed to draw attention and undermine trust.

CSO: What is your strategy for dealing with misappropriated credentials?

Sullivan: We created some great technology modeling the behavior of a real account. It’s machine learning. We have a large group of engineers working on this all the time.

Six hundred thousand times a day, someone tries to log into account using [stolen login information]. We catch them and block them.

I had a meeting with someone from a vendor earlier today, and he told me he tried to log into his Facebook account on the hotel computer [and was presented with a security challenge question]. He was coming from a public computer, and he had the right password. [But] public computer plus different state, that triggered social verification process. Some services would just look at that type of activity as a risky log-in. We do something different, which is social verification. We presented him with profile photos of his friends [and had him pick their names].

We’ve gotten better at this. Using machine learning, we can figure out who are the friends you interact with the most on Facebook.

CSO: How is this learning accomplished?

Sullivan: If I [refuse a friend request], I have now sent two different messages to Facebook about that account. Even in a single friend request, we can see a lot of different ways to learn from that experience. We call it the Facebook immune system. It’s basically scoring every interaction against the site.

Looking [back at the earlier example], what if he was coming from [his home] state but was sending 100 messages per hour, as opposed to last month when he sent 100 messages in all? We can detect that change in behavior. Credit card companies do [something similar.] One time I flew to San Francisco for work, and I had some spare time so I bought my wife an engagement ring. (I didn’t buy her a ring when we got engaged.) So, in San Francisco I used a credit card to buy a ring for her. They called her and got her to validate that it was an acceptable purchase. With the credit card industry, they’re customizing their flags to their environment and the patterns of abuse they see.

Security threats and social media

• The 7 deadly sins of social networking security
• Twitter scoops up mobile security development company
• The final 5 tweets of Harold Wigginbottom, tech-savvy CEO

We have a unique environment, so we have to build a unique profile of [users’] pattern of use and protect against misuse.

When you show up on day one as a new user, you start acting a particular way. If you show up as a fake user, you start acting a different way. So, the best security feature on Facebook is something we don’t have to do; it’s the reporting mechanism we provide for the people who use Facebook. It’s not just that you are a fake user and you send an inordinate number of friend requests to a category of users. You actually also set off alarms to other people.

A fake account thrashes around in the Facebook environment so differently from the way a real person behaves. It’s like the largest and most effective Community Watch program in the world.

CSO: What are some other trust issues you’ve dealt with?

Sullivan: A couple of years ago, the biggest bane of Facebook was half the people loved to use games, and the other half hated hearing about the games. So, we had to respond to that, but we also had to address the underlying issue, which was that [people] didn’t want to spam their friends. For the game developers, the more you spammed your friends, the more successful they were.

Now, if you want to play games on Facebook, you can have that experience without sharing what you’re doing. If you don’t want anything to do with Words With Friends, for example, you have the ability to hide anything having to do with Words With Friends. If my news feed shows one of my friends is listening to a song on Spotify, I can hide it, report the story as spam, or I can dial up or down how much I hear from this friend. Or there is another option, “Hide all by Spotify.” You can make it go away.

CSO: How do you deal with fake accounts created using black-hat SEO tools such as XRumer?

Sullivan: We spend a lot of time and energy on both sides of fighting it. We have a bunch of different relationships to get known bad URLs so we can block those immediately. There’s a Google list, and we’re always getting new lists and not letting people send those links through our service.

It’s detection and mitigation. When the spammers try to target our network, we go after them aggressively. We send cease-and-desist letters. We file civil suits.

Just today, our general counsel appeared in front of the Washington attorney general to go after clickjackers. We try to get criminal enforcement of these civil actions. If you can demonstrate they are using malware and taking over accounts, you can get law enforcement involved.

We have had some success against the Koobface Russian gang. The New York Times ran a story that we looped them in on where we said the number-one type of malware attacking Facebook had been due to the Koobface gang out of Russia. We were able to disrupt their network.

Another one was Sanford Wallace, aka Spamford Wallace. He’s based in Las Vegas. We filed a civil suit against him under the CAN-SPAM Act. We won the two largest judgments since this act was created. The federal judge ordered the U.S. Attorney’s office to open an investigation. That’s very unusual.

We have a “scout” wall in our office. We put up [articles about] our successes in going after individuals who have tried to attack our users. We got an $873 million judgment against Adam Guerbuez. [Editor’s note: Guerbuez was convicted of sending out 4 million commercial spam messages through Facebook.] We localized the judgment in Canada, and we keep moving forward.

[Also see Social media risks: The basics]

There are many different ways to skin a cat. In one case, instead of doing a cease-and-desist, we reached out to the individual’s mother, because it was a teenager living at home. She made him send us a letter and a check. We always ask, What’s the fastest way to make them stop? For that individual, it was through the mother. Every time we have filed a civil suit, the abuse has stopped immediately. It has a chilling effect, even if we don’t ever collect a penny.

No spammer wants an $800 million judgment hanging over their head.

We pay attention to the black-hat forums. They’re talking about what works against this service and what doesn’t work against that other service.

CSO: Do you check in with your counterparts at Google and Twitter to see how they deal with this kind of problem?

Sullivan: We collaborate with them all the time, formally and informally. We don’t compete on security. A rising tide lifts all ships, and that’s the way we all view it.

We defeated Koobface last year, at least temporarily. We have had no compromises on our site since last March. A lot of smaller sites were having problems, so we announced we were going to share this information.

We put together an information packet and shared this information. That’s not a strategy that you use in every case. We get information from other companies, like the bad URL list. We all share those. Or if there’s a situation in which it looks like a large number of email addresses and passwords from one email provider has been compromised, we’ll give them the email addresses from our list so they can require those users to reset their passwords. And they’ll do that for us. It’s a very collaborative environment.

There are a lot of security conferences around the world where companies and researchers get together and talk about trends, approaches and risks. We really value those relationships and we spend time working on them.

We have a Bay Area CSO Council. We have several working groups [with members such as] Google, Yahoo, eBay, PayPal and Salesforce.com. Our security teams are all talking. We have the CSOs all get together and have different topics for a monthly call. We have different sub-groups, an advanced persistent threat group, a compliance policy group. We share best practices. Along with [the National Cyber Security Alliance], we work with companies across the country on computer-safety education.

CSO: Facebook recently changed its interface, rolling out what it calls Timeline. Some users are objecting to this as a violation of privacy. How do you react to that?

Sullivan:We always try to learn and take feedback well; at the same time, we always respect people’s intentions around the audience they want to share with. The Internet used to be divided into people who blogged to the whole world or who communicated in email to just one individual.

One thing I love about Facebook is, when I share something with my friends, I see the interaction with my friends, and they don’t necessarily know each other. We know there is a social validation because they’re both friends with me. That’s what Facebook is all about, that positive interaction with community.

Before the news feed, when you went onto the social network, you saw your own page. You looked at yourself. When it evolved, all of a sudden you had the ability to look at your friends’ profiles. Suddenly, you saw the updates that your friends had done to their pages, and that freaked people out.

It wasn’t a privacy violation because you weren’t being exposed to any more information than you were the day before. It was all there, but it felt very different. The ticker on the right-hand side of the Facebook page is the same thing; there’s nothing there that wasn’t available to you already, it’s just presented in a different way.

Our goal is to get your friends’ actions to you in a way that feels good to you. One of the amazing things about Facebook is, every single user sees a completely different page.

We need to be able to provide a completely different experience for each person. We can’t just dictate it, so that’s where the individual controls come in.

CSO: What is it like being CSO of a one of the most visible companies in the world—and one at which security is continually under siege? Does it get overwhelming?

Sullivan: I started here in 2008. I moved into this role at the end of 2009. I have the best job in the world. I really believe it. Every day brings new and interesting things. If you do contracts as a lawyer, you do contracts no matter where you work. But if you do security at Facebook, it’s very different from doing security anywhere else. We’re dealing with new threats. We’re working at a company that appreciates the value that security brings to the company and the employees. We’re allowed to have an impact. And when we do something, we can see the impact.