When I talk to CISOs or security awareness professionals, I frequently hear the same frustration about the results of their awareness programs. The supposed awareness programs have been a place for a year or more, and they have not yielded noticeable results, and in many cases seem almost useless, as user created incidents seem to continue to increase. When I ask them to describe their programs, what I get are descriptions of components of an awareness program and not a program itself. They describe computer-based training (CBT), and sometimes phishing simulations.[ 5 steps to incorporate threat intelligence into your security awareness program ]As you look to your awareness program, you need to honestly answer, \u201cIs my awareness program working?\u201d More important, you need to ask, \u201cHow do I know?\u201d The second question is actually much more insightful.Answers I heard from people, that I personally don\u2019t believe are valid, include, \u201cI get very positive feedback on the videos used,\u201d and \u201cThe percentage of people clicking on phishing messages keeps going down.\u201dWhen I asked the first person if there were fewer awareness-related incidents, he didn\u2019t know. When I asked the second person if there was a decrease in click-throughs on real phishing messages, I was told there was actually an increase as far as they could tell.CBT and phishing simulations provide easy metrics. That is the primary reason they are common. For example, CBT is generally used to satisfy audit requirements. CBT printouts provide a check the box proof that all employees at least went through mandatory awareness training. It doesn\u2019t prove there was an actual increase in awareness. Phishing simulations usually show that there is a decrease in clicks on the simulated phishing messages. Typically, the same or similar messages are resent until the failure rate goes down. At that point in time, it usually shows that employees recognize the simulations, and basic messages that would normally be stopped by spam filters anyway. There is no inherent proof that people are less susceptible to actual phishing messages.While CBT is unfortunately important due to narrowly defined audit requirements, and can potentially cover a variety of topics, it needs to be reinforced. Cases of organizations having a single person taking the CBT quizzes and sharing the results with other employees, so the other employees don\u2019t have to actually pay attention to the CBT, are common. Phishing simulations, even assuming they are effective, are limited to phishing education, and do nothing to support broader social engineering, physical security, password security, data protection, etc. awareness. It is no wonder that the typical security awareness programs fail.[ 6 essential components for security awareness programs ]While even an organization with a comprehensive awareness program will experience incidents, they should be fewer and more quickly mitigated. In order to have a successful awareness program, there has to be a constant stream of information that is delivered both actively and passively. In some cases, a comprehensive form of gamification can help the organization.The purpose of this article is not necessarily to introduce a new concept, but to address the question of \u201cWhat\u2019s next?\u201d and unfortunately more frequently, \u201cWhy is my awareness program failing?\u201dMy previous articles have addressed those issues. Articles to refer to include:Strategies for creating a comprehensive awareness programHow to provide timely information to your employeesPotential components of an awareness programIncorporating metrics into your awareness programStrategies for creating a comprehensive awareness programIncorporating gamification into an awareness program The good news is that if you have a program in place, whether it is good or bad, comprehensive or not, you at least started to create the infrastructure to move things forward. That is much bigger than it sounds. Security awareness programs can be extremely effective when they are properly implemented, and the effort is more than warranted. So if you are asking, \u201cWhat\u2019s next?\u201d the question itself is a form of success, as you realize that you have either achieved a base level of success, or to paraphrase Thomas Edison, at least you know what doesn\u2019t work.Ira Winkler, CISSP can be contacted at www.securementem.com.