Machine Learning Stops Web Application Threats while Reducing False Positives

istock 957995334

Cybercriminals are increasingly targeting public and internal web applications. Today, nearly half of all data breaches are caused by attacks targeting web application vulnerabilities. To protect your organization from such attacks, Web Application Firewalls (WAFs) are the gold standard. However, some organizations are reluctant to use these devices as they have a reputation for being very resource-intensive, especially when it comes to quickly addressing false positive detections in order to ensure that legitimate users and applications don’t get blocked.

The Limitations of Traditional WAF Threat Detection

The primary reason for the high number of false positive detections generated by most WAF solutions is the underlying core behavioral threat detection method being used. That’s because every single modern WAF in use today relies solely on an observational method for threat detection called application learning (AL).

Application learning (AL) automates the building of profiles for the structure and usage of any web-based applications it encounters. Once enough information is collected, AL then builds policies based on what it has monitored. Subsequent user activities must then adhere to these protection policies or they are identified as anomalies that trigger an action to be taken. These actions can include any combination of logging, alerting, and even the blocking of these detected activities. This detection and response method works to stop sophisticated hackers from attempting to exploit known vulnerabilities or launch zero day attacks.

While this first generation of WAF functionality has certainly improved our ability to identify and respond to web application threats, it still leaves much to be desired in terms of accuracy. Because a WAF solution can generate a high number of false positive detections that can potentially block critical, non-malicious traffic, many organizations have had to dedicate their limited resources to managing policies and exceptions. That’s because there is simply no good way for AL to account for every variation of normal application usage, or to easily adjust to changes in an application, without triggering an anomaly-based filter.

The fundamental problem lies with application learning (AL). Because AL is solely observational, it flags anomalies based only on what it has previously witnessed. This technology simply does not have the necessary intelligence to determine whether a detected anomaly is an attack or is simply benign.

Replacing Application Learning with Machine Learning

Emerging machine learning (ML) technology provides a completely different approach to detecting web security threats. This new approach leverages probability to identify threats rather than running exacting matches against observed activities.

Similar to AL, ML collects data on each application element as users go about their normal application interactions. Unlike AL, however, ML uses a statistical model to determine whether an HTTP request varies significantly from previously observed requests. Only when a request has strayed too far what is considered “normal” does the ML flag that request as an anomaly.

Once an anomaly has been identified, a second layer of machine learning can then be applied to determine if it is a threat or simply a benign variance, such as a typo, a new character that hadn’t been seen previously, or even a legitimate change to the application itself. FortiWeb does this by running the detected anomaly through multiple, highly trained threat models to determine whether a detected anomaly is an attack. If it is, then, just as with traditional WAF solutions, it can take actions such as logging, alerting, and/or blocking the anomaly, but with a much higher degree of accuracy.

To improve its threat detection efficiency even further, FortiWeb includes a variety of specific threat models, each designed for a specific attack category (SQL Injection, Cross-site Scripting, OS Injection, etc.) The AI these threat models leverage are extensively trained and tested using thousands of real attack samples from various sources, including well-known third-party databases such as CVE and Exploit DB data collected through leading third-party vulnerability scanners, and threat intelligence from FortiGuard Labs.

Improved Accuracy and Reduced Overhead

Attack detection accuracy that relies on ML is improved to nearly 100%, especially when using a two-step approach. Instead of flagging and blocking every web application deviation, new machine learning technologies identify and flag real anomalies and then quickly and precisely determine whether they are actually a threat before taking action. This approach ensures that critical applications and transactions are never interrupted. In addition to addressing the false positive detections caused by traditional AL-based WAF solutions, advanced ML engines also dramatically reduce “false negatives,” which are attacks designed to evade WAFs that rely on application learning to detect threats.

Securing application environments presents a unique challenge to IT teams, which is why, according to a recent IDG survey, 83-percent of enterprise IT executives believe that application security is critical to their IT strategy. Whether you are just now considering a WAF for your organization, or are looking to replace an existing solution that consumes too many critical IT resources, consider securing your web application environment with new security tools built around the latest ML and AI technologies.

Read more detailed product information about Fortinet FortiWeb Web Application Firewalls.


Copyright © 2018 IDG Communications, Inc.