Do you use Drupal for your personal website? Does your company use Drupal? Can’t recall the last time it was patched? Well then, as Steve Ragan outlines in this article, it is a safe bet to assume that you’ve already been compromised.
Drupal is an open source software package which was started by Dries Buytaert in 1999. Drupal provides a content management system (CMS) for sites such as MTV, Popular Science, Sony Music, Harvard, MIT and Fast Company as some examples. This software has been the underlying engine of many websites on the Internet today. In fact Drupal is used to power roughly 783,321 CMS websites that are live as of today. This number heads north of 1 million sites running on Drupal that once you take in the entirety of the install base as per Drupal. This puts Drupal in third place behind the juggernaut Wordpress and then Joomla. The large install base coupled with a significant user community behind it makes it an appealing target for users and attackers alike.
A rather significant problem discovered by Stefan Horst came to light recently with Drupal instances. The problem in question is a highly critical vulnerability, documented in CVE-2014-3704, that allows for a SQL injection attack against the database abstraction API. This vulnerability does not apply to every version of Drupal. This problem is specific to versions 7 up to but not including version 7.32. A quick search on Pastebin and similar sites revealed that people were actively hunting down affected sites and sharing exploit code. A successful exploit of a vulnerable Drupal 7 instance could lead to privilege escalation, data leakage, arbitrary php execution. basically just let your imagination run away with you.
For those that fall into the affected category we’re looking at 264,265 live sites that are currently running Drupal version 7, as a CMS at least, as of this writing. The advisory outlining this problem was originally posted on October 15th, 2014. Within 7 hours there were multiple exploits circulating in the wild. A safe assumption that if you are running an affected version that you were compromised unless you managed to have your site updated or patched before Oct 15th, 11pm UTC.
So, what can you do if you didn’t get your patch applied right away? If you don’t have some sort of compensating controls in place like Akamai’s Kona security (full disclosure, I work there for a day job) or similar here is a list of things that you can do to recover your site.
First off I’m going on the assumption that you were running version 7 to 7.32 after October 15th. If so, you’re not safe. There was also a patch that needed to be applied to supplement the 7.32 core. Just running the latest revision wasn’t enough. The folks at Drupal came forward and rather succinctly put it to their users that if you did not apply the patch on October 15th that they are in a heap of trouble.
If you have applied the patch and your site is updated to current you need to be aware that this will not overwrite any backdoors that attackers may have already installed. Bad luck that. Now you need to move into the triage phase. Drupal provided a list of steps that you can take.
- Take the website offline by replacing it with a static HTML page
- Notify the server's administrator emphasizing that other sites or applications hosted on the same server might have been compromised via a backdoor installed by the initial attack
- Consider obtaining a new server, or otherwise remove all the website’s files and database from the server. (Keep a copy safe for later analysis.)
- Restore the website (Drupal files, uploaded files and database) from backups from before 15 October 2014
- Update or patch the restored Drupal core code
- Put the restored and patched/updated website back online
- Manually redo any desired changes made to the website since the date of the restored backup
- Audit anything merged from the compromised website, such as custom code, configuration, files or other artifacts, to confirm they are correct and have not been tampered with.
I just want to expand on these points a little.
1) Document everything. As you go through this process of recovery be sure to keep extensive notes so you don’t forget what you did.
2) First off start by taking your site offline. This will alert any miscreants that may already be on your site but, if that isn’t a concern you can just pull the plug. While this isn’t an appealing prospect you need to take the necessary steps to secure your data as quickly as possible and safeguard your clients / users. You can put up a static HTML page that says something to the effect of “Down for maintenance”.
3) Make a copy of your site. You will need this to conduct a forensic examination of your site or to provide to a third party service to do this on your behalf. If you’re running on a cloud provider such as Amazon this is much simpler as you capture a copy of the instance and save that off for review later. If you are taking regular snapshots you can roll back to October 15th, patch and be back up and running. If you are running on a hosting provider you may need to deal directly with their help desk to get the help you need.
4) Tough decision time. Do you stick your head in the sand and hope for he best? Do you burn it all down or do you take the time to hunt through your website install looking for the indicators of compromise? None of these choices are overly appealing. Hopefully you have been taking regular backups and can roll back to a known good state on or before October 15th. Rollback is far simpler that starting again from a bare metal install.
5) What is your communication plan? Do you alert your customers / users? After conducting a forensic examination of your site this may prove to be necessary if you were in fact compromised. As an example, i you have a single customer in California that is affected you are required to provide mandatory disclosure.
6) Time to roll up the sleeves and dig into the compromised site. That is of course if you have the time, resources or the inclination to do so. A simple tip to keep in mind if that you shouldn’t underestimate your adversary. Imagine that anything was possible and this will allow you to follow the evidence rather than approach it from a confirmation bias. Some easy things to check for are to search out new user accounts that have been added. Check to see if current users have had their passwords changed recently. Compare the files on your system before October 15 with the files that are currently installed. Drupal even provides a module called Hacked! that you an use which will diff the files to find any differences from what was expected.
Here are some other resources that you can use to help:
Your Drupal Site Has A Backdoor - IR process flow
The Drupal Security Advisory - Updated
An FAQ on the issue - From Drupal
Hindsight being 20/20 we can see that the security team at Drupal should have wound up the air raid siren straight away. The original advisory did not hammer home the severity of the issue as well as it could have. It was a shame that it was two weeks before the alarms started to wail. Lessons learned. This has been a rough year for web site owners when you take into account Heartbleed and later Shellshock. Posting a security advisory is helpful to users to be sure but, also sends up a signal flare for attackers. While a co-ordinated disclosure would be a great idea it isn’t necessarily feasible for an install base like Drupal.
This month the vulnerability affected Drupal. Tomorrow it could easily be another platform. Defend your sites. Update and drill your response plans and be sure to have a patch management program that does not amount to “Oh, Fred does the patches."
(Image used under CC from Gabor Hojsty)