How ready are you to stop an advanced attack?

There’s a saying that you can’t improve something you don’t measure. It’s also easy to get seduced into believing you’re good at something by measuring how you perform against a poor simulation of the real thing. There are many different tests you can run to assess your readiness to face an attack. To prevent being lulled into a false sense of security, it is critical to select tests that provide a simulation that is as close as possible to an advanced attack. In this column, I will explore the role that vulnerability assessments, penetration tests and red-team exercises have in preparing you for a real cyberattack.

Vulnerability assessments will give you a sense of how much attack surface you have exposed and how easy it may be to exploit it. The longer a broadly-known vulnerability is present in your environment and the more readily an exploit for it is available, the worse you score on such a test. If the vulnerability is easily reachable from the Internet, your score will take another hit.

Consider the British National Health Service a week before the WannaCry outbreak. An extremely severe SMBv1 vulnerability (CVE-2017-0144) was not patched for at least two months after Shadow Brokers had released an exploit (EternalBlue), which they stole from the NSA. While the vulnerability was not directly exploitable from the Internet, once an attacker compromised any system inside an organization, the exploit provided the basis for a worm to rapidly spread laterally throughout the network.

While vulnerability assessments test the potential susceptibility of your assets to an attack, penetration tests go a step further and actually test the difficulty of an adversary to exploit them. The most common pen tests are tests you can perform on your users and/or your Internet-facing services.

• User tests often involve a degree of social engineering to determine their cyber-risk awareness. For example, how well have you trained your employees to be skeptical of emails that are not what they seem?

You can try out a general phishing campaign, one that involves messages customized to your company or ones that are customized to specific individuals. How do your users perform in the face of your attempts to fool them?

There are a multitude of user tests including getting them to voluntarily start or install software in response to a phone call or text message, or even coaxing them to plug USB keys of dubious provenance into their laptops.

• You can also test out the efficacy of your internet-facing defenses. The main two archetypes for this type of exercise are testing the ability of the defenses to protect your users from the big bad Internet and testing the security posture of your website and other applications which you make available to customers who sit beyond your firewall.

Penetration tests focus on a single step of a cyberattack and thus don’t provide a simulation of an actual advanced attack. For this, security teams within companies generally rely on red teams.

Many breaches which make the evening news involve attackers getting past your initial line of defense, establishing some foothold inside your network, performing multiple cycles of reconnaissance and lateral movement before gaining access to the data they want to steal. Then some combination of data hoarding and exfiltration begins.

Given that pen tests are generally adequate for testing defensive perimeters and are less expensive than red teams, most red teams operate on “assume compromise” or “assume breach” scenarios which start when attackers have already gotten past your first line of defense. To start such a test, the red team is typically provided access to a machine which is assumed to have been previously compromised.

Organizations with larger security teams may have in-house staff that act as red teams while others usually engage third parties to run an attack simulation. The best practice is to use both – one or more in-house red teams to continually probe your ability to defend against a cyberattack and periodic third-party red teams to bring a fresh set of tools and techniques to the attack simulation.

How you perform in the face of well-run red team exercises is the closest you can come to knowing how you will deal with a real-world advanced attack.

But beware of cutting corners when you define the scope of the red team. Don’t give a third-party red team blueprints of your environment – telling them which servers are important and which few systems they can touch gives them a partial map of your environment and limits their movement, neither of which would be true for a real attacker.

In addition, setting an unrealistically short time window for conclusion of the simulation forces the red team to move too quickly and increases the likelihood of tripping an alarm – the real attacker will not have those time limits.

In other words, the more you constrain the scope, the less the red team resembles a real attack. And the less your teams’ performance in the face of such simulated attacks approximates what you can expect when a real-world attack happens.