For years there has been a persistent and spirited debate regarding the effectiveness of information security awareness programs and whether or not they should be utilized within the enterprise.\u00a0 Proponents of such campaigns cite the relative low cost of proactive training versus the high cost of incident response and post-incident remediation efforts, while dissenters often proclaim the unwillingness of employees to actively participate in the information security process or the near impossible task of keeping programs current to address a rapidly evolving and dynamic cyberthreat.\u00a0\tA few months back, Dave Aitel posted an article titled Why you shouldn't train employees for security awareness, offering several fair criticisms of awareness programs and essentially making the case that such programs are almost always ineffective and therefore not worth the investment.\u00a0 While I agree with many of the points brought up such as the alarmingly high rate of cyber victimization among workers (even after education) and the relative success technical controls boast in compensating for human error, I believe the failures of awareness programs are a result of common implementation pitfalls and not inherently characteristic of information security awareness as a concept.\u00a0 Hence, I am unwilling to decry awareness training as a cost-effective and useful tool with a rightful place in any security executive\u2019s bag of tricks.\u00a0 Instead, I offer a handful of reasons why information security awareness programs fail, none of which I believe are uncorrectable, and some of which complement the innovative notions set forth by Joe Ferrara in his article 10 commandments for effective security training.\tReal-world exercises are absent or insufficient\tInformation security awareness programs often have a solid educational component, but lack exercises to effectively test the employees\u2019 ability to retain information or recognize the real threat.\u00a0 Other programs include such exercises, but \u2018dumb them down\u2019 or fail to include those individuals who serve as higher value targets to adversaries.\u00a0 For instance, if conducting a mock phishing exercise, it is the first instinct of many experts to make the phish easy to recognize and to avoid sending it to upper management.\u00a0 The justification is that the former will yield positive performance metrics and provide management a warm and fuzzy, while the latter will avoid making waves among C-levels.\u00a0 In both cases, the purpose of the exercise (to train and prepare) is largely lost.\tTraining points to processes that are immature or unreliable\tMany programs make a concerted push to inform employees of process changes and foster compliance with new requirements (e.g. report incidents to someone else, begin logging off on this day and time to receive updates).\u00a0 The problem lies in that often times the employee is exposed to the awareness material before the process is broadly implemented, tested, and mature.\u00a0 The result might be an employee following instructions, only to find out that the person on the other end of a call is unaware of their responsibilities or a technical staff member fails to follow-up on a report or follow through with a process. \u00a0These incidents harm the credibility of the program, and can propel participants into a permanent state of resistance.\tContent is either not customized or it\u2019s excessively customized for the enterprise\tContent should strike a balance between the general and the specific as well as the practical and the conceptual.\u00a0 Far too often, security awareness programs are based around general threats that fail to hit home with the intended audience.\u00a0 This may simply be because little is known about the specific threats facing the organization.\u00a0 In other cases, specific incidents and internal metrics serve as the only input to the development of awareness materials, resulting in training that is insufficient in addressing organizational change.\u00a0 As a general rule of thumb, awareness programs should adhere to a three-tiered model.\u00a0 One third of the content should be based around general information security practice, one third is based around industry or sector-specific standards or threats, and the last third is based around the organization\u2019s posture and the targeted threats it faces.\u00a0\tFormat and delivery are not adequately diverse\tThe best programs contain a diverse set of awareness materials including print, email, and web-based content as well as face-to-face interaction with employees, the last being the most often overlooked component due to the cost of time and travel.\u00a0 The fact is that employees learn in different ways and will seek out their desired format.\u00a0 If it\u2019s not available, then they simply won\u2019t buy in to the program.\u00a0 If it\u2019s not communicated persistently and frequently, they will ignore it. Further, opportunities to provide both physical and electronic takeaways such as desktop wallpapers, corporate merchandise and prizes, or even candy will foster participation among those who might typically ignore infrequent messages regardless of their purpose.\tMaterial is one size fits all or lacks a personal element\tEvery role within an organization holds a different set of responsibilities, and information security awareness material should reflect that.\u00a0 If your IT director and the mailroom clerk receive the same training, then it\u2019s likely at least one of them is getting little to no value from it.\u00a0 Content should be driven by the employees\u2019 role, their security know-how, their exposure to technology, and the level of privileges they are granted in the course of their duties.\u00a0 Enterprises often lack specialized awareness material for those with administrator privileges on their desktop, or those that carry mobile devices or smartphones.\u00a0 Awareness material should address individual activities, particularly those that are considered to be high-risk.\u00a0\u00a0 Further, as pointed out by Audrey Agle in Seven practical ideas for security awareness, personal engagement is paramount.\u00a0 It is likely that most employees have home networks, wireless devices, children using the Internet, or engage in personal banking or investing using the web.\u00a0 Provide them with tips to keep them and their kids safe \u2013 it will encourage participation and buy-in, and will garner their appreciation and support for the program.\tSample sizes are insufficient or do not represent the organization\tObtaining quantifiable data and gauging employee performance is critical.\u00a0 Unfortunately, generating an awareness metric for every employee in a large enterprise is at best costly, and at worst impossible.\u00a0 As a result, experts frequently rely on sample sets to assess performance.\u00a0 These samples can skew results on multiple fronts, the first being size.\u00a0 There are statistical measures that will help you formulate an appropriate sample size for exercises, surveys, and the distribution of awareness materials. Another pitfall is a sample that does not reflect the organization as a whole.\u00a0 This can result from oversampling certain business units or employee roles, or better yet sampling only English-speaking employees in order to cut the costs of translating awareness training and broadening metrics gathering.\u00a0\u00a0 The solution \u2013 make certain that samples are constructed at random, and that everyone has an equal likelihood of both receiving material and being used as a performance metric.\tCommunications are not championed and compliance is not enforced\tMost people recognize the need to champion a cause from the top down in order for it to be effective.\u00a0 Regardless, security awareness materials are often relegated to the \u2018back page\u2019 of company training portfolios and upper management is noticeably absent from security awareness communications.\u00a0 Further, the absence of management from the program makes it difficult to enforce information security policy violations and instances of non-compliance.\u00a0 This is particularly true as change is introduced and new processes are rolled out.\u00a0 The fact is in order to be effective, employees must be exposed to real consequences for ignoring, resisting, or rejecting awareness materials and content.\u00a0 I once had an executive suggest that bonuses be withheld from those employees who failed to complete training.\u00a0 While that type of Draconian technique might keep human resources and legal personnel up at night, there are lighter alternatives such as refusing to issue devices until training is complete, temporarily revoking administrator privileges, or notifying supervisors and next level managers until someone takes notice.\u00a0\u00a0 \u00a0\tPast incidents are ignored for the sake of confidentiality\tCorporate fears of bad press, declining stock prices, and a loss of goodwill or consumer confidence are certainly warranted and well placed.\u00a0 As a result, executives do everything they possibly can to prevent potentially damaging information surrounding security breaches and\/or incidents from making their way to the public.\u00a0 This often means that past incidents become taboo and are not discussed in any form, particularly awareness materials.\u00a0 This has a detrimental effect on security as the most teachable moments are lost in the ether.\u00a0 A more constructive (and still safe) approach is to sterilize the information associated with an incident and use it to train employees to recognize the actual threat.\u00a0 How does one do this without divulging damaging information?\u00a0 Simple \u2013 keep the discussion focused on the attack vector, not the result of the incursion.\u00a0 Rather than discuss technical vulnerabilities and IT response measures, identify only the component where employee action (or inaction) would result in a positive outcome.\u00a0\tOutside vendors are used ineffectively\tEnterprises may look to outside vendors to support their security awareness campaigns or formulate exercises to test employee knowledge and preparedness.\u00a0 Sometimes, this can be beneficial, as outside vendors tend to have specialized expertise and can provide an unbiased, independent perspective about your security posture.\u00a0 However, often times specific requirements are not discussed beforehand, resulting in a boilerplate program that does not yield much needed metrics and falls well short of being customized to your organization, industry, and risk appetite.\u00a0 Avoid wasting time and money by identifying clear objectives prior to the work being done.\u00a0 Further, be weary of relinquishing total control of content development and the execution of training exercises to an outside vendor.\u00a0 Instead, leverage a blend of outside experts with internal personnel who are intimately familiar with your environment and the type of metrics upper management demands.\u00a0\u00a0\u00a0\tAbout the Author\tSalvatore C. Paladino is a Cyber Security Analyst and Project Manager with a large defense contractor in support of the Department of Homeland Security and the Department of Defense. His areas of expertise include technology evaluation, transition, and deployment, information security policy development, information assurance training and awareness, and the identification of emerging cyberthreats. He has authored numerous technical papers and has testified before the New York State Commission of Investigation as an expert witness specializing in cybersecurity.\tMr. Paladino holds a BS with a concentration in Computer Security from Utica College of Syracuse University and an MBA in Technology Management from the State University of New York. In addition to being certified in Risk and Information Systems Control (CRISC) and a Certified Information Systems Security Professional (CISSP), he is a CompTIA Network+, Security+, A+ and CTT+ Certified Professional. He is also an Adjunct Instructor of Cybersecurity in the School of Business and Justice Studies at Utica College.\tSal can be reached for comment at email@example.com.