Independent security research and testing firm NSS Labs today released its most recent Network Intrusion Prevention System (IPS) Comparative Group Test Report for the fourth quarter of 2010. The previous NSS Labs network IPS report was released in September 2009. In that study, NSS Labs found that security effectiveness ranged from a dismal 17.3 percent to a high of 89.5 percent.
Many of those failures a year ago resulted from the failure of network IPS vendors to stop techniques used by attackers to simply evade the defensive properties of IPS security gear, explains Rick Moy, president of NSS Labs.
Since that time, NSS Labs has found significant improvements:
- Security effectiveness, using the default factory-shipped settings, rose to 62 percent. But be careful: some default settings reached a mere 31 percent effectiveness.
- The improvement in security came with a price: performance of these devices decreased overall. One vendor, says Moy, reached only 3 percent of its advertised throughput.
- A number of multi-function gateways rose to comparable effectiveness as dedicated network IPS gear.
- Tuning is required, adding an average increase of 21 percent more protection.
Security equipment gear from Check Point, Endace, Fortinet, IBM, Juniper, McAfee M-8000, NSFOCUS, Palo Alto Networks, Sourcefire, and Stonesoft were tested.
HP TippingPoint refused to participate in the study, Moy says.
The products were tested using nearly 1,200 live exploits under what Moy describes as real-world conditions. Each device was tested using the default settings from the vendor, then once again more finely tuned by a representative of the respective vendor.
In the test using the manufacturer's default settings, McAfee's M-8000 came out on top, with 92 percent effectiveness, while the IBM GX6116 faired the worst at 31 percent effectiveness. Security effectiveness changed dramatically once devices were tuned. In those tests the Sourcefire 3D 4500 scored best, at 98 percent. And, according to the report, the Endace Core-100 came at the bottom at 43 percent.
NSS Labs charges $1,800 per user for the report, and has requested that full results not be published.
The report shows that not only shouldn't enterprises rely too completely on the ability of an IPS to protect their network, they should expect to spend considerable time maintaining the device. "It's not out of the ordinary to spend a few days a month tune it," says Moy. Who adds that the amount of time users have to spend tweaking their device is proportional to how well the detection signatures are written.
Most importantly, the report details how little trust users should place in data sheets and they should thoroughly test any network IPS devices they're considering.