Black Hat SEO, part two: SEOwN3d!!1

In part one of our series on the collision of search engine optimization and black-hat hacking (see “Black Hat SEOs: Is This the Future of Search?”), we explored how search engine optimizers, or SEOs, have learned tricks that change the search results that drive much of the traffic to successful websites. (The practice of search engine optimization is also called SEO.) Many of these upstart entrepreneurs have made small fortunes as SEO consultants. Many also use SEO to drive traffic to their own sites that sell products, ads or referrals–a business known as search marketing.

We explored how the tactics of SEO include some unsavory ones that range from digital ffibs to aggressive deception. The tricks are called black-hat SEO, though that’s something of a misnomer since, as SEOs like to say, they don’t break the law, just the search companies’ terms of service. The search companies tried to stay ahead of black-hat SEO by tweaking their algorithms and adding filters that penalize sites for questionable tactics. Increasingly, though, it looked as if the combined forces of SEO and black-hat hacking would be too much for any algorithm….

As search companies have tried to contain the more aggressive techniques that SEOs were using to manipulate search-engine rankings, black-hat SEOs have responded by circumventing the rules. Rather than just using loopholes, they began actively abusing the algorithms used to determine search engine results. The tactics became so aggressive that the SEOs started to make the search engines look bad: Search results started to reflect the SEO’s reality, rather than a reality that rewarded good sites. Like all arms races, this one eventually escalated to an untenable level. The game had to change again. And it did, about 18 months ago.

Suddenly and without much warning, search companies–Google especially, the SEOs say–decided to enforce its terms of service, and severely. The algorithms wised up some, but more than that, it appeared that Google was buttressing its algorithm with filters and manual labor. If enough complaints came in about a site using black-hat tactics, Google would manually adjust the rankings or simply blacklist the site–a process SEOs call a “hand job.”

Some SEOs and search marketers were surprised. The top SEOs generally maintained good lines of communication with Google and other search companies. Some, like Jeremy Schoemaker–a search marketer known online as Shoemoney–would even periodically ask for advice on SEO techniques and whether they’d get him in trouble.

But now the search companies were matching the SEOs’ aggressiveness. The effect could be devastating. A site that was blacklisted lost its traffic, and therefore its business, overnight. Usually targeted sites clearly violated search terms of service. But some weren’t doing anything differently than they’d been doing for months or years. “When people are ranking for a phrase and supporting their family, and then the next day they’re off the map, that’s really vicious,” says Schoemaker. “You can literally ruin someone’s life.”

Of course, Google could make the argument that turnabout is fair play. Perhaps enforcement was brusque and arbitrary, but so is black-hat SEO. Nothing Google was doing was illegal, which was an argument the black-hat SEOs had made for years. Plus, as early as 2006, Matt Cutts, Google’s chief liaison to the SEO community, had blogged about the ramp-up in enforcement against overly aggressive SEO.

Even before that, the veteran SEO Eric Ward warned others that eventually the free ride would end. Ward was notorious for his cautious, by-the-book approach to link-building strategies. Some called him “a poser,” “arrogant” and “retarded,” and bestowed him with the nickname “Linkmoses.”

“I understand why [the search engines] are doing it, but their enforcement has become a little heavy-handed,” says SEO Michael Gray. Says Aaron Wall, another SEO: “Google went on a crusade.”

The Aftermath of the Crusade

As frustrating as delisting was for companies suddenly punished by SEO enforcement, getting relisted proved to be a much worse problem. SEOs and site owners found themselves stuck with little communication from the search companies about what they had done wrong or how to fix it to get back in the good graces of the algorithms. Schoemaker himself lost the top spot for ring-tone searches.

“I was making thousands of dollars a day, and then one day I was out of Google,” he says. “I inquired why and never really got an answer. They said it was normal search engine fluctuation”–fluctuation, he notes, that also can be caused by black-hat SEOs. “I probably got gamed out,” he suggests. He currently ranks about tenth in ring tones.

Google also partnered with Stopbadware.org to blacklist sites that were potentially infected with malware. Last September, a Web-hosting company in Thailand was hacked and several sites that used the host were flagged on Google, so that if users clicked on a link to the site, an intermediate screen popped up warning them that the site they were about to visit was potentially infected.

Obviously, people rarely visit a site after that kind of warning. The owner of the hosting company, Daniel Peterson, says that after he had cleaned up the sites, nothing had been done to get those blocked sites relisted in Google search results. “No one seems to want to do anything, and the blacklisting is now seriously damaging our businesses,” Peterson wrote in an e-mail.

He is particularly concerned about a boutique hotel in Pattaya called Rabbit Resort. Peterson wrote: “Rabbit Resort seriously relies on their Google listing and normally receives 50 to 60 visitors every day. Most of these become bookings. They now receive one every day or so. With more than 60 staff to employ, they now risk financial ruin and disaster.” (The sites were eventually relisted).

Roger Thompson, the blogger for Exploit Prevention Labs, cites another recent case, in which search results for “saints football club” brought up a number of Australian soccer team sites that were labeled as potentially containing malware. Thompson notes that another site had this happen. “K1-usa.net…used to be the number-one organic result when people searched for k1. They were hacked for about 10 days, and then cleaned, but in the meantime, they had earned the ‘This site may harm your computer’ label, and over the next 12 months, before the label was removed, their rating slipped, and slipped, until finally it was nowhere on the first three pages.”

Most of the soccer sites were marked clean within days, not months, suggesting Google has improved in the relisting game. “We can always try to do better,” says Cutts, the Google liaison. “We’re trying to be as responsive as possible.”

But Thompson notes, “This happens quite a bit, and I must admit that I’m surprised no one has accused Google of damaging their brand.”

“Our webmaster guidelines are clear,” says Cutts, who noted that Google made this policy in anticipation of problems with sites using others to goose their rankings. “We say that ultimately you are responsible for what’s on your site. If the scam is on your page, that’s what is causing damage. We’ll do whatever we can to try to help, but ultimately if there’s spam content on your pages, we’re willing to remove that content, and then hopefully cycle that back in when it’s cleaned up.”

RSnake, a security expert with experience in Web advertising and SEO who runs ha.ckers.org, says that no matter how blunt and overzealous enforcement has become, that’s not the problem with Google’s approach to enforcement. It’s that the policy that Cutts is referring to is ultimately faulty, because it’s based on a false premise.

“Google can shut you down at any time,” says RSnake. “But there are all kinds of weird things that could happen to you, upstream problems, a proxy goes bad, someone takes over your site, and there’s no way for you to explain that it might not be your fault. They’re making false assumptions about how the Internet works, which is that the owner of the IP address is always in control of what happens through that IP address.” (Indeed, some black-hat SEOs seized on the opportunity and complained about competitors’ sites in hopes that they could get them manually pushed out of the rankings.)

Still, Google’s policy of flagging sites and aggressively delisting any site using black-hat SEO remains in place, and by January of this year, Ward felt vindicated for his conservative approach to SEO. About the crackdown on black-hat SEO, a gloating Linkmoses (he has embraced the nickname) wrote a blog entry, “Don’t Blame Google for Your Linking Failures”:

In 2007, many long-practiced link building tactics stopped being effective. Many link building companies and consultants sold the exact tactics/services that are now useless. Why didn’t you see this coming, and if you did, why did you sell those services in the first place and what services will you sell now?… Are you really going to tell me you are shocked that Google no longer thinks a link from link-o-matic, link-to-my-loo, and LinksForNoGoodReason.com are of any value? Please. But if you knew that such links would someday lose value, why did you take money for that very service? And if you didn’t honestly know such links were pointless, how can you call yourself a link builder? Google’s focus on trusted sources is your worst nightmare.

The Devil They Didn’t Know

Certainly gray techniques are still being used by SEOs, and they always will be; Schoemaker recently uncovered a ring-tone business that had come up with a way to take up all the Google AdWords paid-links results for any given search. He estimated that the scheme could net $1 million in four months, and he was surprised Google hadn’t banned the company yet.

Still, the crackdown has had an effect. It appears to be cleaving the business. Many SEOs are going more white hat, if you will, and a few have decided to go full-out black hat–a phenomenon that security researcher Jeremiah Grossman calls “SEOwN3d!!1”, a mash-up of SEO and hacker slang for compromising a site.

Some decided that the free ride was over, and they cleaned up their act. They’ve adjusted to the new rules of the playground. The noted SEO David Naylor gave up black-hat SEO and even abandoned jobs for which his revenue would be based on traffic volume. Instead he works on retainer and consults for flat fees–trading in the potential for periodic obscene windfalls for a less outrageous, more stable income. “If I slip off that first page, I still get paid now,” he says. “And I’ve got a team of guys I’ve got to feed. It was a total business decision.”

Cutts of Google believes this is the primary trend. “I primarily see growth in white-hat SEO. Most are savvy enough to know that they can’t afford to be delisted. The industry as a whole is heading toward white-hat SEO.” But he also concedes the point that hackers and SEOs “are getting a little more affiliated, and more SEOs are delving into that world.”

They’ve cleaved the other way, crossing into the realm of the illegal to keep the game going. If Google won’t let black-hat SEOs build link farms or stuff comments fields with links, then they will exploit legitimate sites and use them as cats’ paws in their schemes. Of course, an early target has been .edu domains. “Almost all of the .edu hacks now are for SEO,” says RSnake. “Not just a few of the big hacks. I mean almost all of them.” Domains with .mil extensions, which also pass “juice” (SEO lingo for tactics that increase Web rankings), are targets now, too.

Primary entries into sites are XSS, SQL injection and FTP vulnerabilities that allow strangers to manipulate the site. Hackers traditionally used those vulnerabilities to insert bots on a site for distributing spam, stealing personal data or some other scam. Now they are being used to stuff links on the page. They hide the links by making them the same color as the background (an old technique for keywords made new) or by simply cloaking them, so that the spiders see them but people do not.

If the site gets good traffic–like Al Gore’s ecology blog–those hidden links get good juice. Another scam uses the bots to give redirect commands that send browsers to link farms. Recent headlines illustrate this: “Forth Road Bridge hack redirects to smut bazaar” and “Perl.com sends visitors to porn link farm.” Many SEOs said hacking and surreptitious linking are rampant on social networking sites, and blog platforms like WordPress (where Al Gore’s blog lived) are under constant attack as hackers look for high-traffic zones to plant their links and their bots.

Another illegal technique a bot might be used for is cookie stuffing. Here’s one cookie-stuffing scheme: Around tax time, a hacking SEO uses compromised sites to secretly inject cookies onto the computers of site visitors. On those cookies are referral links to the tax prep websites. If my machine had been stuffed with one of those cookies, the person who put it there would collect a referral fee when I signed up to use one the tax prep sites.

Many experts believe this is only the beginning and that, because there’s so much money to be made off the search business model, the techniques will get more sophisticated and far more clever. “From my point of view,” says Grossman, “it’s just getting started.”

Even Linkmoses held no illusions that Google’s crackdown would eliminate black-hat SEO. “Enforcement means higher rankings will go to creators of truly awesome content, and bad guys,” he says. “It’s been a game of leapfrog since day one. There won’t ever be a time when people won’t game the system.”

David Naylor believes that black-hat SEO has gotten so good that search itself is being devalued. Trust has eroded. “You type a search into Google and believe what comes back in the number-one slot is the truth, and it’s not,” he says. “It’s often some version of the truth engineered by very clever people trying to make a lot of money.”

The SEOwN3d!!1 Effect

Just as auguries’ decisions about the observed flight patterns of birds reverberated through Rome, affecting religion, the outcomes of wars and the fate of rulers, so too do the effects of SEO schemes ripple across the Internet–affecting how SEO is used, what it’s good for and what it will look like in the future.

As SEO migrates to illegal activity, the primary effect is the collateral damage it creates. A report from Websense, the Internet filtering company, estimated that 51 percent of sites hosting malware now are legitimate sites that have been compromised, and many of those are compromised for SEO and search marketing schemes.

A simple cookie-stuffing program illustrates the havoc SEOs and search marketers can create. Cookie stuffing involves the illegal access of an innocent site, which is then used to serve illicit code to customers without their knowledge, based on their arriving there through a search engine. Meanwhile, a company is paying referral fees to search marketers who haven’t earned that fee while possibly taking those fees away from people who had earned them but whose legitimate referrals were overwritten by the cookie stuffer.

So those who run the hacked site are mad at the hacking SEO. Customers are mad at the hacked site and at the search engine for bringing them to a hacked site. A company is mad about paying money to someone who didn’t earn it, while someone who should have earned it is mad at the company and the other hacking SEO.

Where there’s collateral damage, there’s litigation. A few lawyers have started looking at the space as possible fertile ground.

“It’s quite possible that the next few years will see some lawsuits against providers that allege the use of SEO tactics,” writes James Grimmelmann in an Iowa Law Review article from last November about the ambiguous state of search engine law. While he notes some challenges of suing based on SEO, he also notes, “Courts have recognized that some techniques of content design are deceptively manipulative and cause harm to legitimate providers, and it is possible that innovative pleading could properly state other business torts against manipulators. Similarly, luring users to one’s content through SEO raises significant false-advertising concerns. In these cases, competitors, users and consumer-protection agencies might all be proper plaintiffs.”

But that’s speculation. Naylor, among others, says that aggressive and illegal forms of SEO have already had more tangible effects on the Internet and what it’s good for–or rather what it’s no longer good for.

“One of the things black-hat SEOs did, and did very, very well, was to go into Web landscapes and just destroy them,” says Naylor. “I mean, at one time people liked having guest books on their sites, and SEOs just filled them with all these links to the point they became unusable. Now why would you have a guest book? It’s asking for trouble. Why would you let people put comments on your blog? Are you crazy?”

The optimizers are changing what’s valuable online, by changing what looks valuable because it ranks high in a search. Black-hat SEOs, and now hacking SEOs, are so good at their craft that they force search companies to constantly change the algorithms and filters. The factors that give a site juice are in some ways the ones that SEOs haven’t yet exploited.

Some SEOs argue that no online feature exists that they won’t be able to game. What black-hat SEO demonstrates, they say, is that the search algorithm isn’t magic at all. It’s just software that, once understood, is easily outwitted by humans.

The Men Behind the Curtain

To deal with this, the SEOs believe that the search companies have deployed humans of their own–rooms full of them–whose job is to essentially buttress the algorithms’ decisions with human ones. “They have to keep this mystery algorithm looking like it’s working correctly,” says Schoemaker. “So they have all these places around the country where they hire humans to hand-edit results” that have been affected by black-hat and hacking SEO, he says.

“They don’t say it openly but I’ve read enough from Matt Cutts and others to know that this algorithm they purport does everything magically, it’s all a bunch of nonsense,” says Dave Dellanave, Schoemaker’s partner. “The reality is they have probably thousands and thousands of filters that they manually create. And there’s no doubt in my mind that increasingly they’re using people, the ‘human signal,’ for rankings.”

Critical SEOs contend that this is the only way the search companies can protect their indexes from widespread abuse by black-hat and hacking SEOs. “They’re trying to protect their index,” says SEO Michael Gray, “because if it’s clean, people want to use it, and if people want to use it they can sell advertising. The lower value the search results, the less valuable to users and advertisers.”

Cutts says that the “vast majority” of ranking (and of reconsideration requests when a site is delisted) is “algorithmically done.” He also contends that “Google is returning more relevant search results in the last year or two.”

But critics argue that “relevant” is in the eye of the beholder. The phrase used in the industry for the new direction of search companies is a focus on “trusted and authoritative links.” But what makes something trustworthy or authoritative, especially when the search engine can’t intuit what a person is looking for to begin with?

Many SEOs say that “trusted and authoritative” is code for “big, well-known company.”

“The real direction of search,” says SEO Wall, “is that large corporations will dominate search results, and they’ll get away with more aggressive SEO because the search engines can’t afford to look bad by not having them at the top of results. You’re more likely to get enforced against if you use aggressive SEO if you’re smaller, not bigger. Small companies will not be able to compete through search.”

Many of the SEOs compared this to big-box stores driving locally owned independent stores out of business in small towns. Search results would become dominated by large brands that can afford to keep themselves atop the rankings and that the search companies consider trusted and authoritative, because they’re well-known.

This, the SEOs say, is finally where black-hat SEO is driving general search, and now hacking SEO is as well. It’s turning the Web into a big strip mall.

Scott Berinato is former executive editor of CSO. Send feedback to Derek Slater at dslater@cxo.com.