Protocol analyzers: Dos and don’ts

As with any tool, protocol analyzers need skilled implementation. Here is advice from the front lines.

Read more in the companion article Protocol analyzers: How to choose and use them (no registration is required).

DO make sure you have the right expertise on your network operations and security teams to make effective use of protocol analysis tools to troubleshoot network problems, tune firewalls and other security devices, and investigate the cause of attacks on the enterprise.

In a large organization, chances are there are IT people with the right credentials to make effective use of these tools, but they are thin on the ground. “A lot of network admins would be stunned by what they see in Wireshark,” says Opus One’s Joel Snyder. Generally, look for experienced network engineers or security personnel with a strong hands-on background configuring network firewalls and intrusion-protection systems.

Your tech folks should have a thorough understanding of protocols and how they work, so they can quickly examine the packet captures, determine where the issues are and remediate them. Experienced pros can apply this knowledge to security and application issues as well as network operations.

“What you really need is someone who is pretty well rounded—think of the OSI model,” says Mike Chapple, senior adviser to the executive vice president at the University of Notre Dame. IT people are often specialists: programmers who specialize in the application layer or network operations staff who know network and transport layers, for example.

“If you’re troubleshooting, you have to very quickly and agilely navigate up and down the protocol stack, from the physical layer straight up the application layer,” he says.

If you don’t have sufficient expertise with protocol analysis to cover your IT operations effectively, consider bringing in consultants, especially for major projects that involve introducing new services or for forensics after a major security incident.

DO invest in enterprise versions of protocol analyzers or higher-level monitoring and analysis products in a large, distributed environment in which your analyzer jockeys are thin on the ground.

“The skill gap is increasing, and there’s a big shift from the scenario in which you arm techs with protocol analyzers and parachute them into remote offices,” says NetScout’s Shalita.

Enterprise-caliber appliances allow you to capture and store traffic data across your networks, perform at least some automated analysis, and troubleshoot from anywhere in the enterprise. Your networks are big, fast and complex. Make the most of your available personnel to keep your networks and applications running smoothly with minimal disruption.

DON’T sell protocol analyzers short. Make their use an integral part of your network, application and security procedures.

“People use Wireshark as a last resort,” says Riverbed’s Combs, the tool’s creator. “Packet capture seems to be the last thing people do. A lot of times they may not even have a span port set to capture traffic.”

DO establish baselines of what constitutes normal network traffic and application behavior so you can quickly and accurately assess where the problems lie, inform the CSO or other appropriate manager what’s going on, and follow the appropriate escalation and remediation procedures.

[Also learn more about Network behavior analysis tools]

“People don’t baseline,” says Combs. “So they have a problem when they fire up an analyzer and have no idea what’s good and what’s bad.” He strongly recommends having a set of capture files taken on a normal day. “You can pull it up and see what you need to focus on. It’s a big timesaver.”

DO consider high-end application performance and network monitoring and analysis products. These aren’t cheap, and they require strong policies and procedures so you can quickly assess risk based on the information they present and then take the appropriate action. But they can be of considerable value, particularly when you consider the potential impact of interruptions of key applications and services. Several products monitor and capture everything that moves across your networks, increasing your ability to identify intrusions and other security events that might go undetected for days, weeks or even months.

These tools also associate users with network and application activities, so you have the ability to identify who may be involved in unauthorized or malicious behavior. Forrester Research recommends a zero trust model of information security, a departure from the traditional “trust but verify” adage. Ubiquitous visibility into network and application traffic gives you the ability to monitor, report and, if necessary, act on anything your users are doing.

DO leverage network and application analysis to coordinate operations among network, application and security personnel. At a tactical level, share findings of protocol analyzer activity when they cross areas of responsibility. Enterprise-level tools allow groups to conduct their analyses and act on a common data set, rather than engage in redundant activities. In a time when resources are thin and workloads enormous, the lines are blurring and cooperation, not finger-pointing, should be the order of the day.

“Every time I see a healthy security organization, one of the main driving factors is that they have a good relationship with a strong network organization—they’ve become so intertwined it’s hard to separate the two,” says Chapple. “Management needs to set the tone, but the real relationship happens at the engineer level.