Security vendors should face the music, even if they hate the tune

Two firewall vendors are fuming over a NSS Labs report that questions the effectiveness of their products. But fuming in public is the wrong response.

The trouble started with news that independent IT security testing company NSS Labs had evaluated six network firewalls: Check Point Power-1 11065, Cisco ASA 5585, Fortinet Fortigate 3950, Juniper SRX 5800, Palo Alto Networks PA-4020, and the Sonicwall E8500.

As George V. Hulme wrote, the results were not encouraging:

What the company found would likely startle any existing or potential customers: three of the six firewalls failed to stay operational when subjected to stability tests, five out of six didn’t handle what is known as the “Sneak ACK attack,” that would enable attackers to side-step the firewall itself. Finally, according to NSS Labs, the performance claims presented in the vendor datasheets “are generally grossly overstated.”

I can understand how such a verdict would send vendors reaching for a few stiff drinks and a few more curse words to describe NSS Labs.

Nothing hurts more than when your integrity is called into question.

But when it comes to the public response, there’s a useful way to go about it and a not-so-useful way.

The firewall vendors seem to be following the latter approach.

As my Network World colleague Ellen Messmer wrote:

A test by NSS Labs that found firewalls from five vendors are subject in one way or another to remote exploit by hackers has ignited furious response from vendors Fortinet and SonicWall.

“NSS Labs tested the Fortigate-3950B platform using equipment supplied by a NSS customer and not configured by Fortinet,” said Patrick Bedwell, vice president of marketing at Fortinet, in a prepared statement. Bedwell’s remarks go on to say that Fortinet was “not given the opportunity to work with NSS Labs on the testing” but that “we have been working diligently with NSS Labs over the last month to remediate any issues raised in the test.”

The Fortinet statement says “the FortiGate platforms are not susceptible to split handshake attacks when AV [antivirus] and IPS [intrusion-prevention system] engines are enabled, which was suggested to NSS as the initial solution. In addition, following guidance received from NSS’ CTO, Fortinet developed new IPS signatures to explicitly block the handshake, which are available today to all customers. Lastly, Fortinet agreed to implement changes in our firewall functionality to explicitly block the split handshake after learning that NSS didn’t consider IPS signatures as a valid response for this particular test.”

“They said we failed the test,” says Dmitri Ayrapetov, SonicWall’s product manager for network security, explaining why SonicWall is upset with the report from NSS Labs. He adds SonicWall has a checkbox-activated feature that can be turned on to address the TCP split handshake security issue, and that SonicWall repeatedly “asked them to turn it on” and change the box from the default setting.

The NSS Labs report does point out the existence of this SonicWall checkbox-activated feature.

Now, I don’t take issue with these vendors publicly challenging the results. What I do take issue with is the tone.

“Fortinet was not given the opportunity to work with NSS Labs on the testing…”

“SonicWall has a checkbox-activated feature that can be turned on to address the TCP split handshake security issue, and SonicWall repeatedly asked them to turn it on…”

Comments like these are whiny. Whiny never helps your cause. It turns off your current and potentially future customers.

The better approach would have been to acknowledge the issues raised in the tests in a matter-of-fact manner and simply go on to explain how the issues can be managed now and in the future.

Perhaps the testing techniques were flawed. It doesn’t matter. When your effectiveness is called into question, whining is the last thing you should do. As I tell my kids all the time, whining gets you nowhere.

It’s better to just say you are looking into the claims made in the test, you have steps customers can take in the meantime, and promise a fuller response later.

Customers are more forgiving when you do it that way.

For the record, I think both vendors put out high-quality technology. I’ve worked with both companies in the past for various articles, and they have always been helpful.

So instead of getting even more uptight about what I’ve said here, they should just take it in the spirit I intended:

Constructive advice from an old friend.

–Bill Brenner