Enterprise Security Moves Toward Intrusion Prevention

Targeted hacker attacks on enterprises have been increasing, and are generally launched by more sophisticated and motivated attackers. Intrusion prevention is gaining importance while intrusion detection is fading away. Gartner has defined three key criteria for true network-based and host-based intrusion prevention.

Intrusion Detection Has Problems

As enterprises became disenchanted by the performance of intrusion detection products, security vendors stopped using the word “detection” and began re-labeling their products as intrusion prevention or intrusion protection. However few products provide the features that Gartner believes are necessary for true intrusion prevention.

In the post-Internet boom, with the dawn of Web services and with the network becoming an integral part of business operations, security solutions such as intrusion prevention systems are being introduced that offer real protection for the enterprise. Similar to intrusion detection, intrusion prevention can be separated into two broad categories – host-based and network-based.

Mandatory Requirements for Intrusion Prevention

An intrusion prevention system must meet three key criteria:

• It must not disrupt normal operations. When it is inserted online, a network-based intrusion prevention system must not introduce unacceptable or unpredictable latency into a network. A host-based intrusion prevention system must not use more than 10 percent of a system’s resources. Normal network traffic and host-based processes should operate identically, whether an intrusion prevention system is running or not. Blocking actions must occur in real time or near real time, with latencies in the range of tens of milliseconds (not seconds).
• It must block malicious actions using multiple algorithms. Intrusion prevention systems must provide blocking capabilities that include signature-based blocking of known attacks. However, intrusion prevention systems must also move beyond simple signature-based approaches – such as those used by antivirus and intrusion detection systems – to at least support policy, behavior and anomaly-based detection algorithms. These algorithms must operate at the application level in addition to standard, network-level firewall processing.
• It must have the wisdom to know the difference (between attack events and normal events). As intrusion prevention systems mature, they will be able to positively identify and block higher percentages of attacks than today’s first-generation intrusion prevention systems (that is, firewalls) do. However, they will never be perfect, and it will always be necessary to flag suspicious activity for further human investigation. Thus, the intrusion detection market will be relegated to mere first-alert status.

Host-Based Intrusion Prevention

Host-based intrusion prevention is software that resides on a server and prevents cyberattacks against the operating system or applications. Products from Okena and Entercept Security Technologies have had early success in protecting servers, particularly against the “Code Red” and “Nimda” attacks. Host-based intrusion prevention is an immediate cure for vulnerabilities in servers, but because of the costly overhead of managing security software on many diverse platforms within the enterprise, host-based intrusion prevention systems will not see the same adoption rate as network-based intrusion prevention.

Host-based intrusion prevention technology can apply policies based on predefined rules or learned behavior analysis to block malicious server or PC actions. Host-based intrusion prevention can stop attackers from implementing buffer overflow strikes, changing registry keys, overwriting Dynamic Link Libraries or engaging in other approaches to obtain control of the operating system.

Host-based intrusion prevention can be implemented as software shims that intercept calls between applications and the underlying operating system, or as kernel modifications that apply more-stringent security controls than those built into commercial operating systems.

Examples of software shims are the following:

• Network Associates/Entercept Security Technologies
• Cisco Systems/Okena
• Sana Security
• GreenBorder Technologies

Examples of kernel modifications are the following:

• Argus Systems Group
• Sun Microsystems’ Trusted Solaris Operating System
• Hewlett-Packard’s Virtual Vault

Host-based software that simply locks down the host and only allows certain applications to execute does not meet Gartner’s criteria for host-based intrusion prevention, because it does not protect against flaws in permitted applications.

Network-Based Intrusion Prevention

The advantages of network-based intrusion prevention systems include the reduced importance of constant monitoring, and that an attack does not set off chimes and claxons that cause a chaotic scramble to react. Network administrators know that Code Red attacks have become part of the background radiation of the Internet. Therefore, the time spent logging and responding to such attacks is wasted. Once identified, the affected session should simply be dropped. Thus, not only are valuable resources conserved, but also a better overall security posture is achieved. The defining characteristics and benefits of network intrusion prevention are:

Firewalls and gateway antivirus systems are examples of first-generation, network-based intrusion prevention systems. However, firewalls primarily operate at the network protocol level, and antivirus systems largely implement simple, reactive (that is, non-real-time), signature-based detection and blocking.

A true network-based intrusion prevention system must:

• Operate as an in-line network device that runs at wire speeds.
• Perform packet normalization, assembly and inspection.
• Apply rules based on several methodologies to packet streams, including (at a minimum) protocol anomaly analysis, signature analysis and behavior analysis.
• Drop malicious sessions – don’t simply reset connections.

To do all that, network-based intrusion prevention must perform deep packet inspection of all traffic, and generally must use special-purpose hardware to achieve gigabit throughput. Software-based approaches that run on general-purpose servers may be sufficient for small enterprise use, and blade-based approaches may scale up to some large enterprises. However, for complex networks running at gigabit rates, Gartner believes that application-specific integrated circuits and dedicated network security processors will be required to perform deep packet inspection, and to support blocking at wire speeds.

Vendors include:

• TippingPoint Technologies
• IntruVert Networks
• NetContinuum
• iPolicy Networks
• Fortinet

Characteristics and Benefits of Network Intrusion Prevention

The defining characteristics and benefits of network intrusion prevention are the following:

• Inline position: Rather than tapping into a data stream from the switch or other device, the products sit inline with the data stream. Inline systems can analyze and identify packets and sessions, verify which are malicious and drop the associated stream of packets. That is essential to the products’ protective abilities.
• Stateful signature: To efficiently handle multi-gigabit traffic streams, some form of stateful inspection must be used. The state of a particular communication going over a network includes the ability to have session knowledge about the packets being analyzed. Some awareness of state enables the engine to parse only the pieces of the session that are applicable to the attack signature. That provides high throughput and low latency, which are also required for enterprise applications.
• Combined algorithms: No single methodology can catch the maximum number of intrusion attempts while minimizing false positives. Intrusion prevention systems must use a combination of methodologies:
  • Signature analysis is the most powerful method, but it must be augmented with protocol/packet anomaly detection.
  • Protocol/packet anomaly detection focuses on signatures within the protocol or packet that have been defined as hostile, malformed, out of sequence or potential “zombies,” which are some distributed denial-of-service (DDoS) relay kits that can serve as transmitters for floods of packets to be sent to DDoS targets’ servers. The relay kits use Internet Relay Chat channels to communicate back to controlling hackers, who can direct the relay kits to start attacking certain Web sites. By bombarding sites with bogus traffic, hackers can make it impossible for a site to respond to legitimate connections.
  • Behavior-based statistics are less exact, but they can provide a valuable function. This technique involves analyzing baseline metrics of known traffic patterns, then setting the alert threshold when extreme traffic pattern changes occur, such as massive flooding that may indicate a denial-of-service attack. (Flooding may also indicate a legitimate network traffic surge. Thus, notification can maintain or alert required infrastructure changes to meet valid traffic demand.)
• Dropping malicious traffic: Once a malicious session is identified, it is simply dropped, which protects the destination server or device. Logging and alerting are functions of these devices.

Intrusion Prevention Summary

Some facts about intrusion prevention are the following:

• Firewalls are intrusion prevention devices.
• As intrusion prevention begins to include application-level attack blocking, products must meet a minimum set of criteria before enterprises can take them into consideration.
• Intrusion detection will always be required to give warnings about activities that are suspicious but not necessarily hostile.
• Most enterprises will require hardware-based intrusion prevention products to protect high-speed networks.

Bottom Line

• As processing power and security algorithm performance increase, intrusion prevention will grow in importance, while intrusion detection will shrink.
• However, through 2006, enterprises should deploy a combination of both capabilities to meet security best practices.

This article is an excerpt of a chapter from a new report, “Securing the Enterprise: The Latest Strategies and Technologies for Building a Safe Architecture.” The report is an offering of the Gartner Executive Report Series, a new business venture of Gartner Press that provides buyers with comprehensive guides to today’s hottest IT topics. For information about buying the report or others in the Executive Report Series, go to www.gartner.com/executivereports.