Twelve Reasons Pen Testing Won't Die

Core Security Technologies CTO Ivan Arce lists 12 reasons Fortify Co-Founder and Chief Scientist Brian Chess is wrong about 2009 marking the end of pen testing

Each year, during the sprint to the finish line of the fourth quarter, just before we have our last chance to revisit our carefully thought out and meticulously prepared and negotiated plans for the next year, the distinguished group of modern Nostradamuses in our industry unveil their penultimate prophecies for the year to come.

Don't get me wrong. The art of security industry futurology is not for the faint of heart and I deeply respect the artists. To predict not only what will happen but also when and how long it will last, a hefty amount of courage and intrepidness is required. Perhaps that is why the pros in the business equip themselves with a series of mathematical modeling gimmicks and an arsenal of surveys and statistics to corroborate their chosen hypothesis.

If you are interested in becoming one of them but feel that you are not brave enough, fear not! There is still hope for less daring individuals -- such as myself -- willing to become apprentices at the School of Information Systems Security Prophets (SISSP).

To get to the entry level, the Security Prophet's Minor League, so to speak, one only needs to predict something that will not occur in the next year and get it published in a respectable medium.

Yes! This sounds easy, doesn't it? I've entertained the idea that I could do it and in the past weeks I've prepared some very suitable predictions along these lines: "In 2009 Web applications will not get rid of the security bugs that plague them" or "In 2009 applying security patches will not become obsolete" or, the even more audacious: "In 2009 firewalls will not be deprecated."

At first this seemed easy but then I realized that those predictions would not suffice to pass my SISSP examination and even if they did getting them published somewhere respectable would remain an open issue.

Fortunately, I found an opportunity I could not pass on -- this week's CSO Online article covering a 2009 prediction from Brian Chess, CTO of Fortify, a colleague for whom I have much professional respect: Penetration Testing: Dead in 2009.

As you probably already guessed, in this article I intend to counter-predict Brian (don't get upset Brian, I know you've been extremely gracious quoting me in your book and sending me a complimentary copy and this is how me, the Most Ungrateful, pays you back. If I was even remotely decent I would be ashamed of myself and not do this, but what can I say: The temptation was too great and I am a weak).

So, yes, indeed, I hereby predict that "2009 will not mark the end of pen tests as we know them." In fact, since I am feeling quite bold right now, I will double down and predict that "2009 will not mark the end of pen tests as we don't know them," either.

Of course, now I am required to provide a plausible rationale for such bold statements, so if you're still reading bear with me.

First I will need to clarify what penetration testing is as I know it.

Penetration testing is an information security practice that is at least 35 years old, and its origins can be traced back to the Multics Security Evaluation performed by the US Air Force in 1974 and IBM's Penetrating an operating system: a study of VM/370 integrity from 1976. It is often -- mistakenly -- considered an arcane and costly infosec practice due to its perceived dependence on highly-skilled and specialized professionals that can successfully model the mindset and modus operandi of real attackers.

Contrary to somewhat popular belief, pen testing is not a simple "badness-o-meter" that can only produce two outputs: "broken security" or "don't know."

As a time-constrained exercise of realistic testing of an organization's security posture using an attacker's perspective, penetration testing provides a much richer output that can be encoded with a single bit. It highlights actual, specific weaknesses that a set of modeled attackers with a given set of skills could combine to generate successful attack paths that realize threats to the organization within a specific timespan.

The natural (and expected) result of a penetration test is a detailed account of the set of attacks (all of them) that were preformed that could and could not successfully breach the organization's security, their respective impact and a corresponding set of actionable items to address them. Bearing that in mind, I will now attempt to justify and rationalize my counter-prediction to Brian's prediction.

Penetration tests will not die or suddenly turn or become assimilated into something else in 2009 because:

  • 1. A 35-year old practice with steadily increasing adoption rates does not usually disappear or transform itself substantially within just 12 months.
  • 2. Penetration testing is intrinsically operational in nature. While pre-emptive measures such as security QA and testing and other SDLC practices may be useful to reduce the number of security vulnerabilities in custom or newly developed software, existing operational environments will continue to have bugs during 2009 due to the deployment of legacy or un-audited buggy applications.
  • 3. Penetration testing is operational in nature (did I say that?). It deals with multistage and multilayered threats or attacks (not just vulnerabilities!) in real-world environments (not test labs) and then maps them explicitly to actual security risks. This will remain a valid use case scenario during 2009.
  • 4. Penetration testing is tactical. It provides tangible, actionable information on how to incrementally improve an organization's security posture effectively to prevent real and specific attacks from happening and do so efficiently since it makes it easier to measure at least some for of return on security investment considering both the defense and offense technology currently available.
  • 5. Penetration testing is strategic. If performed regularly, consistently and as part of an organization's overall security strategy, it becomes a useful and valuable practice to implement a program of constant improvement of information security.
  • 6. Penetration testing is strategic. Incorporating an attacker's perspective to an organization's overall security strategy provides necessary checks and balances and improves the organization's ability to steer security policy in accordance to current trends in the threat landscape.
  • 7. Penetration testing is not a silver bullet. It is best used in conjunction with other security practices and in doing so it amplifies those results with both positive and negative feedback (about what does and does not work).
  • 8. Penetration testing is -- at least partially -- driven by compliance. It is a recommended or even a mandatory practice in several regulations, industry standards and organization's internal policies that will not go away in 2009.
  • 9. The IT landscape is constantly evolving and will continue to do so in the next year. As new technologies emerge, new attack vectors become prevalent. Monitoring the evolution over time of sophisticated penetration testing techniques is a good leading indicator of threats that may see mass-adoption in the future which makes pen testing almost a necessity to improve SISSP qualifications.
  • 10. There is money to be made selling penetration services and products. The opportunity will not go away in 2009.
  • 11. Financial crisis and economic turmoil means also more and better opportunities for cybercrime. In the context of 2009 testing one's defenses periodically will be more (not less) necessary than if we had a more globally stable scenario.
  • 12. Last but not least, five years ago IDS technology and its respective market was the "in thing" to make predictions about. Although predicted several times, the death of the IDS has been greatly exaggerated in the past years.

It appears that in 2008 penetration testing achieved enough relevance to merit an apocalyptic prophecy and pre-emptive obituary. That by itself may be enough evidence to attest that pen testing has now shadowed the IDS star and may enter adulthood at 35, knowing that what did not kill it made it stronger.

Insider: How a good CSO confronts inevitable bad news
Join the discussion
Be the first to comment on this article. Our Commenting Policies