Service levels for security systems – a call to arms!

Would you like to buy a spam filter that is “pretty good”?  How about a biometric authentication system that is “generally right”?  I know, would you be interested in an intrusion detection system that primarily flags valid network activity?  Of course you wouldn’t, right?  Well, the sad news is that many businesses have purchased such systems.  The fact is, security technology is frequently sold with very minimal or no real performance commitments.  Service levels, if provided at all, are so qualified that performance guarantees are virtually useless. 

An example, a large, national company recently deployed a very expensive, state-of-the-art spam filtering system.  The business is reliant on its ability to process, respond, and store all client e-mail.  The problem is that the system was only 98% accurate.  That means, out of every 100 e-mail, 2 valid client e-mail, on average, are falsely identified as spam.  Well you say 98% is very good, but what about the 2 valid e-mail flagged as spam?  With this system, the user must go to an intranet site to view e-mail filtered as spam and move any incorrectly flagged e-mail back to their in-box.

Again, this doesn’t sound like much of a problem.  But consider this, since the users can’t rely on the filter being completely accurate, they had to check the intranet site on at least a daily basis to look for valid e-mail incorrectly flagged as spam.  A smart user finally used a stopwatch to time how long it took to complete the foregoing.  Then she timed how long it took to simply directly delete spam as it came into her in-box (i.e., she turned off the filter and just dealt with spam as it arrived).  Imagine her surprise when she discovered using the expensive spam filtering system actually took more of her day than simply deleting spam the old fashioned way.  Her discovery led to a wave of other users disabling the spam filter on their systems as well.  Because of the lack of user interest and the loss in employee productivity, the company ultimately uninstalled the very expensive spam filter they had licensed.

 I’m not saying anti-spam systems and other security technology aren’t extremely useful and absolutely mandatory in most instances.  What I am saying is that businesses need to think about the performance levels they really need to make the system useful and insist on those protections in their agreements with security vendors.  Of course, no system can be 100% accurate and no one is saying they must be.  What they should be capable of doing, however, is provide a level of guaranteed performance that justifies the investment (both monetary and in human costs).  Many security vendors are very sensitive to this issue and have worked hard to offer strong protections and guarantees in their agreements.  Those that don’t are in for a wake-up call.