Pentagon contractor spied on social media, left data unsecured in cloud

Not even the Pentagon is immune to shoddy security by a dreaded third party. Recently, a researcher discovered three unsecured Amazon Web Services (AWS) storage buckets that contained at least 1.8 billion scraped social media posts that were collected as part of global, military-sponsored web monitoring program. Oops.

The misconfigured AWS S3 buckets, discovered by Chris Vickery of UpGuard, were labeled centcom-archive, pacom-archive and centcom-backup. CENTCOM refers to the U.S. Central Command, and PACOM is short for U.S. Pacific Command.

Unsecured AWS S3 buckets had ‘dozens and dozens of terabytes’ of collected social media posts

Vickery told The Register he found the CENTCOM archive while scanning publicly accessible S3 buckets for the word COM.” The three unsecured buckets contained “dozens and dozens of terabytes” of collected social media posts and other content posted on the internet.

Unsurprising, the scraped social media posts included posts in Arabic, Farsi and dialects spoken in Afghanistan and Pakistan. But the three exposed buckets also included social media posts made by Americans.

“The repositories appear to contain billions of public internet posts and news commentary scraped from the writings of many individuals,” according to UpGuard. At least 1.8 billion were posts scraped from the internet over the last eight years, “including content captured from news sites, comment sections, web forums, and social media sites like Facebook, featuring multiple languages and originating from countries around the world. Among those are many apparently benign public internet and social media posts by Americans, collected in an apparent Pentagon intelligence-gathering operation, raising serious questions of privacy and civil liberties.”

Some of the details collected included web addresses of the posts, “as well as other background details on the authors that provide further confirmation of their origins from American citizens.” Facebook and Twitter were apparently popular targets for scraping, with some posts stating political opinions, but UpGuard said “everything from soccer discussion groups to video game forums are sources for scraped web posts” were included in the vast repository.

“Massive in scale, it is difficult to state exactly how or why these particular posts were collected over the course of almost a decade,” UpGuard said.

A couple examples provided by UpGuard included a post from Poker Fraud Alert Forums about boycotting Trump’s companies and another from Debate Policy. The latter scraped comment featured a quote by the fourth U.S. president and founding father, James Madison, about Americans’ liberties followed by a citation from the Bible.

Inside the “scraped” folder, there was also a folder labeled “Coral,” which UpGuard said “likely refers to the U.S. Army’s ‘Coral Reef’ intelligence software.” That folder included a directory called “INGEST,” which contained all scraped posts held in the “centcom-backup” bucket.

While the “scraped” folder contained internet content scraped from 2009 to 2015, the CENTCOM bucket contained data collected from 2009 to present day.

“The most recent indexed files were created in August 2017, right before UpGuard’s discovery, consisting of posts collected in February 2017,” UpGuard said.

Lax security by government contractor VendorX

All of the data was collected by the now-defunct government contractor VendorX, which did work for CENTCOM via a project called Outpost. That program was described as a “multi-lingual platform designed to positively influence change in high-risk youth in unstable regions of the world.” That doesn’t explain why publicly posted content by Americans were included in the program.

VendorX was entrusted with gathering all this data, but it couldn’t be bothered to make the servers private. The flip side is that we never would have known about this web monitoring program for the Pentagon had it been secured. VendorX left the settings wide open so that anyone with a free AWS account could have browsed and downloaded the data.

“A simple permission settings change would have meant the difference between these data repositories being revealed to the wider internet, or remaining secured,” UpGuard wrote. “If critical information of a highly sensitive nature cannot be secured by the government — or by third-party vendors entrusted with the information — the consequences will affect not only whatever government organizations and contractors that are responsible, but anybody whose information or internet posts were targeted through this program, potentially resulting in unfair bias or unwarranted actions against the post creator.”

Pentagon denies the collection was for intelligence purposes

The Pentagon denies the scraped data was part of a military-sponsored intelligence gathering operation. A spokesperson told PC Mag that the contractor collected the data using “commercial off-the-shelf programs” and it was reportedly “not collected nor processed for any intelligence purposes.”

Perhaps it goes to show that anything you say online can come back to bite you — or at least end up being collected by a government contractor for internet surveillance.

Maj. Josh Jacques, a spokesperson for U.S. Central Command, confirmed that the S3 buckets were exposed, saying to CNN, “We determined that the data was accessed via unauthorized means by employing methods to circumvent security protocols. Once alerted to the unauthorized access, CENTCOM implemented additional security measures to prevent unauthorized access.”