• United States




Penetration testing is a reference point, not a strategy

Jan 22, 20184 mins
Data and Information SecurityIT StrategyNetwork Security

Pen tests are valuable only if the results are properly translated into an effective overall security strategy.

network security digital internet firewall binary code
Credit: Thinkstock

I’m often skeptical of survey results, but a recent survey from the 2017 HIMSS (health sector) conference, which suggests that penetration testing is a top priority, caught my eye. Add to this Gartner’s global cyber security group estimate of a 14 percent uptick in “security testing,” as well as an 8.5 percent increase in “consulting.” Combined, these projections imply that many organizations may be readier to prioritize budget towards penetrating testing. That’s great, if we understand the limits and the proper role of pen testing an overarching security strategy.

Never trust a test that says you’re perfect

At a conceptual level, penetration testing is a great way to help manage security in an organization. It’s great at evaluating the efficacy of your security system’s detection and response mechanisms, and it’s a fantastic tool when you want to build a case to affect meaningful internal change and/or ask for a larger security budget.

But a penetration test that ends up showing that your defenses are impenetrable is likely fatally flawed.

I was a penetration tester back in the day (when it was much easier), and having worked with scary-good penetration testers here at Critical Informatics, I can tell you they never lose. They always get in.

There are several common phishing pretexts that almost always work: a request from the “boss” to review a document, holiday gift cards, changes to the dress code policy, or a donation-match form. The payload will either request some action of the user, such as logging in to a web page with their domain credentials, or will be embedded in an attachment like a Word document, Excel Spreadsheet, or PowerPoint presentation.

Once the payload has been executed or “detonated,” long-term persistence is sometimes established in the form of user auto-runs. Then it’s a matter of reconnoitering the network until administrative credentials are obtained, and then pivoting around the environment until something worth stealing is found. With little variation, penetration testers look for the main authentication database and then go to work cracking passwords.

So, what should we take away from this reality?

  1. Threat actors out there are using these same techniques, and they’re just as successful as penetration testers
  2. The main target for initial penetration attempts is your people, not your software
  3. Spending more on preventive controls and employing practices like application whitelisting can impede attackers, but they won’t leave you airtight

So yes—you’re secure until the moment you hit the radar of someone who has, or who can acquire, what are becoming commoditized skills. If penetration and other testing of your defenses is something you’re prioritizing this year, be aware: the information you will obtain is not revelatory, and simply addressing the specifics of whatever vulnerability was exploited will not appreciably change the outcome for the next penetration test (which may not be a test).

After the test

A penetration test can help drive internal messaging, especially around the need for resources to address the ease with which the information technology can be compromised. It can help to prioritize on actualized business risk rather than fear, uncertainty, and doubt. In other words, the vulnerabilities identified are not as important as the management outcome.

However, there are two areas of focus that should be evaluated against the results of a penetration test:

  1. User training: There’s no firewall for stupid, but you can move that needle in your direction through mitigating controls like secure configuration baseline, user training, messaging, carrots for appropriate performance, and perhaps the occasional public shaming.
  2. Detection and response: The BEST way to leverage the results of a penetration test is to evaluate how well the activity was detected, and whether your response was adequate to curtail the activity and prevent actual loss in the future.

Remember—penetration tests are valuable as a reference point, but only if the results are properly translated into an effective overall security strategy. That way, when your ticket is punched, you’ll be ready.


Michael Hamilton, is the founder and CISO of CI Security, formerly known as Critical Informatics, a provider of managed detection and response and information security consulting services.

With 25 years of experience in information security as a practitioner, consultant, executive and entrepreneur, Michael has worked with Fortune 100 companies to small private colleges, and in nearly every sector.

As former Chief Information Security Officer for the City of Seattle, Michael managed information security policy, strategy, and operations for 30 government agencies. Prior, Michael was the Managing Consultant for VeriSign Global Security Consulting, where he provided his information security expertise for hundreds of organizations.

Michael is a subject-matter expert and former Vice-Chair for the DHS State, Local, Tribal and Territorial Government Coordinating Council. His awards include Member of the Year with the Association of City and County Information Systems (ACCIS), and Collaboration Award from the Center for Digital Government.

Michael recently served as a Cyber Security Policy Advisor for the State of Washington Office of the CIO, and continues to spearhead the Public Infrastructure Security Collaboration and Exchange System (PICSES), a regional cyber event monitoring system that is unique in the nation. Michael has been a member of the Sigma Xi research honor society for more than 25 years.

For the latest in cybersecurity news, follow Mike on Twitter at @seattlemkh and CI Security (@critinformatics).

To see more from Mike, check out his articles and videos on the CI Security blog.

The opinions expressed in this blog are those of Michael Hamilton and do not necessarily represent those of IDG Communications, Inc., its parent, subsidiary or affiliated companies.