What’s in your test, training, and exercise program?

May marks the month of graduation for many college students across the country. For the past few years, these learners have been testing and training in a classroom environment, and now they are presumably ready to enter into the ‘real world.’ Are they prepared though?

If you question whether college grads have acquired enough hands-on experience to aid in putting out the security fires blazing in your enterprise, then perhaps you may also want to question how well your test, training, and exercise program prepares your security practitioners to respond to a crisis. 

The National Institute of Standards and Technology (NIST) developed standards and guidelines not only to aid corporations in developing a preparedness plan but also to ensure that the plan is maintained in a state of readiness. The NIST guide said, “This includes having IT personnel trained to fulfill their roles and responsibilities; having plans exercised to validate their policies and procedures; and having systems tested to ensure their operability.”

But are these plans ensuring that exercises are not overly focused on narrow types of sensational threats, to the point of yielding little value for the types of threats more likely to affect businesses?

With the current tactic of posing red teams against blue teams, said Ben Cianciaruso, co-founder and COO at Verodin, “There is way too much emphasis on prevention and not a lot around detection and response. Exposing holes doesn’t really facilitate improvement and learning.”

[ MORE ON CSO: Earn your black belt through free training  ]

Verodin has noticed in many organizations that a lot of people were heavily reliant on the tools that they had. “What we are trying to do is enable organizations on the defender side with something that is measurable to mature those capabilities,” Cianciaruso said.

In examining the testing exercises of different enterprises, Cianciaruso said, “One of the things we found is that red teams are set up as a gotcha exercise. They are intended to show the blue team ‘look we got you’, but it doesn’t really provide a mechanism to allow the blue team to improve. That is where you are going to get your value.”

Rather than identifying holes and playing the ‘gotcha’ game, Verodin wants to see organizations rethinking their training and testing exercises so that the red team is instead able to hand over their findings to the blue team and re-run attack simulations so that now they have the means and the access to learn and improve.

“Understanding where the failings are is critical in preparing them for when the attack happens for real,” said Cianciaruso.

The goal in these testing, training, and exercise programs is to get more value out of the people you have, to “Measure stock abilities and provide opportunities to address where they failed on a quarterly basis. To see whether you are improving through these exercises,” Cianciaruso said.

What people are doing now, though, is setting up a separate network for training, but they are not able to do anything within the production environment. It’s all hypotheticals, Cianciaruso said. “If this happens, this is what we do next. There is no real means to fully understand that these are the exact alerts and actions. This is what I will see on the screen. It’s all tabletop exercises even if you are calling them functional exercises,” he continued.

Because understanding risk is critical to being ready to respond, the more you can do it in a real environment, the better positioned you are to put out the fire before it rages out of control. Perhaps it’s time for your security team to graduate to a new test, training, and exercise program.