How to Corral Security Consultants

Security consultants can help your business, if you give them clear ground rules before they start

My current boss, a CEO, defines a consultant as a person you pay to tell you what time it is from your own wristwatch.

I like that line. Having been on both sides of the game, as a consultant and a customer, my view is that definition is sometimes right on the money. While there are some very good security consultants out there, and some very good customers, they don't necessarily communicate very well with each other. And that opens the door to problems (and, of course, to consultant jokes). I suppose if your intent is to gain outside confirmation of your own beliefs, hiring a consultant can be useful. But be prepared for the possibility that the consultant may return with an opposing view or advice you don't think your company would be wise to follow.

A few days after I landed my present job, post-9/11, I was told that one of my performance objectives was to track the progress of the security consultants who had been hired and launched before I got here. They were brought in to "look things over and make recommendations to improve security." Once they were through looking and we had their reports, I was to review those reports and develop plans to implement the recommendations. Sounded reasonable. Within days, however, I learned that things weren't quite that simple: There wasn't just one security consulting group on board; there were three, and all were nearly finished. Each had a slightly different approach, background and number of team members. Each had been hired by operations directors from different departments to perform a "comprehensive review of security," but those hiring managers didn't coordinate their efforts with each other or with the consultants. And, not being security professionals, the ops directors did not think themselves qualified to place any restraints on the consultants, which meant that, with no useful guidance from our end, the consultants pretty much had complete freedom.

In our case we pretty much created a monster. The ops directors who hired these experts had nothing but good intentions. But they gave the consultants too much freedom.

That may sound bad, but wait, there's more. These consultants had mostly Defense Department experience and little background with the private sector, which meant they had no sense for business planning around P&Ls. When their reports came in it was no surprise to see that they were in different formats, that they contained both different and overlapping findings, and that they made different recommendationseven in cases where the findings were the same!

Too Much Advice Can Be a Bad Thing

I had expected some duplication. But I was not prepared for the labor-intensive development of a matrix, first to identify the more than 600 findings and recommendations, and then to decide which were different, which were duplicates and which were contradictory. The net of all this was assimilating a little more than 100 separate findings and recommendations that may have made sense to the military, but did not translate very well to the private sector.

And here is the most irksome issue when hiring security consultants. They will walk away when finished. You, however, have to live with what they leave behind. In the best of all worlds, you took the time to properly orient them to your business, your culture, your standards and your needs for their help. Then, of course, you made it clear where you are on the risk management scale. If your business is risk averse, like the Department of Defense, you may not bat an eye when you get the recommendation for a high-efficiency particulate air (HEPA) filter. But if your business is more accepting of risk, more in the mode of managing various risks, then you will want to see some more practical approaches to things like air quality assurance.

If you haven't managed the consultant's goals and objectives and have not placed any constraints on them, you will find, as I did, that you are caught between the proverbial rock (the consultants' professional opinion) and a hard place (your reality). The key, therefore, to a successful relationship with security consultants is to clearly define what you want to achieve through their service, when you think it is reasonable for them to finish, what constraints your company imposes on any business proposal, and finally, what format their final report should be in. You'll also be ahead of the game if you request that, for each recommendation the consultants make, they engineer the solution, cost it out and draft the budget justifications (in terms of ROI). That way you'll be prepared when it comes time to fight for the money. Also, have them provide an estimate on the impact to the annual operating budget of maintaining all the systems and gadgets they recommend you buy.

In our case we pretty much created a monster. The ops directors who hired these experts had nothing but good intentions. But they gave the consultants too much freedom. This led to some equally well-intentioned recommendations that created a number of "round peg/square hole" problems for us. While 90 percent of their work was common sense and not controversial, we were quickly reminded that the final 10 percent of performance accounts for 90 percent of the cost. So when I decided to ignore some of their more outrageous suggestions I had to do a lot of homeworksome of it with the help of even more consultantsto prove that doing it their way was either silly or, in my view, flat-out wrong. Remember the old proverb that says the more you pay for something, the more credibility it has? Exactly the case here.

When one of the ops managers said, "What do you mean you aren't going to follow their recommendation to install HEPA filters in our public building HVAC systems?" I had to explain exactly what a HEPA filter is (and its impact on standard HVAC design and our operating budget) before he stopped sniveling and listened. He had allocated a large slice of his discretionary budget to pay the consultant, so he expected us to follow their advice. All of it. But I pointed out that in a public building a bad guy can simply walk in and spread bad things around that circumvent the filters. And since the best defense is to turn off the HVAC system to keep bad things from spreading, the filters are once again largely irrelevant. Not to mention state building codes on the replacement of air in public buildings make HEPA filters impractical in an existing HVAC system.

Even with my (I thought) lucid arguments, we unfortunately ended up hiring an engineering consultancy to study the impact and cost of installing respirator filters in a building HVAC system. The resulting study showed conclusively that it didn't make any sense. The consultants may as well have recommended we put canary cages in the food court to warn us when we are under chemical attack! In the end we paid the engineering consultant a lot of money to tell us that the security consultants had made a silly recommendation that we were right to ignore.

Guide Events or Be Guided by Them

So, OK, what is the lesson here? Never hire a consultant? No, of course not. There are times when you will want to do this, no matter how good you are yourself. But before you go that route, make sure it is a deliberate decision and that you have a big hand in shaping the course of events.

First, it's important to understand the difference between a security professional and a security practitioner. A security professional may be certified by a recognized security association and have many years of experience in security but not be currently responsible for the security of any enterprise. A security practitioner is someone who is responsible for all or part of the security of an enterprise, whether or not he has any expertise in security at all. The best case is when the practitioner is also a professional, and the professional has been a practitioner.

With that in mind, it's appropriate to hire a security consultant:

  • When nobody in the company has the requisite expertise.
  • When there may be a legitimate question of conflict of interest. If you were the champion last year of an unpopular security policy change, that change will be hard for you to look at without bias this year.
  • When, regardless of expertise or conflicts, nobody has the time to do it.

If you find yourself in one of these situations, here are some things to look into when selecting a security consultant:

  • Get recommendations from professional contacts in your industry whom you know and trust. If they were happy with a consultant, chances are good you will be too.
  • Require consultants to submit team member résumés with their proposal. You should look for senior team members who have been security practitioners. Since a lot of these people come out of the Defense Department or a police department, look for recent experience in your business sector.
  • Call some former clients of the consultant and talk to them about their experience with this company; find out what they liked and disliked about the service they received.

Before any work commences, make sure you get formal nondisclosure agreements signed by each person involved in the security work. Also be sure to schedule a formal meeting where you set mutual ground rules for your work together, and schedule frequent status reports. Make sure you get to see the final draft before it goes to anyone. You should have no surprises at the formal presentation of consultants' findings or with their written reports, and you should be able to anticipate and answer questions from your boss and other higher-ups. This isn't to suggest a whitewash; simply, it's best that your peers and your leadership hear about security problems and solutions from you rather than from the "experts." After all, you hired these people to help you get better, not to make you or your company look bad. Following this advice will enhance your experience with consultants, and ensure you present your company with a useful set of recommendations that improve security without breaking the bank or harming your credibility.

Copyright © 2005 IDG Communications, Inc.

How to choose a SIEM solution: 11 key features and considerations