Threat detection: it’s about ‘time’

Incident response is a slave to time. From time-to-detection through time-to-containment, time is the crucial factor when responding to any threat.

eye security privacy time clocks deadline
Thinkstock

Incident responders have no shortage of products that help to identify threats. From the earliest virus scanners and IDS products to modern solutions that utilize machine learning to analyze behavior, the goal has always been simple yet unattainable: to keep adversaries from accessing private resources and doing damage to the organization. While there is no specific right or wrong way of trying to achieve this, there is an unspoken principal that is critical to this process is “time.” Time factors into each step of the incident response process, from time to detection through time to containment.

Let’s look at three major categories of threat detection to show how time effects each. First, we will look at endpoint solutions, network detection and then log analysis. As Einstein has shown us, time is relative. As a point of reference, we will refer to the moment of compromise as being “n.

From the beginning to the endpoint

Endpoint solutions are frequently the last line of defense before the associated device is breached. They potentially employ a number of techniques in order to identify and block threats before they are provided access to execute on the protected device. These techniques can range from relatively simple (checking files for known threats) to complex and resource intensive (sandboxing untrusted applications). Some solutions also record detailed transaction information for later forensic analysis (which we will cover later). 

Endpoint solutions must operate within certain User Experience boundaries in order to be truly effective. For a solution to perform blocking, it must be able to make a blocking decision within milliseconds, otherwise the security solution may introduce too much lag and noticeably effect the performance of the protected device. Thus, endpoint enforcement operates in the area of around n – 10ms. Products that slow down the endpoint or utilize too many resources are frequently disabled by the user, and a disabled security solution can sometimes be even more dangerous than a non-existent one. An endpoint can only employ protection that can utilize rapidly enough to not impact endpoint performance noticeably.

An endpoint solution is critical, as it can protect assets regardless of location. While it is good to be able to block a threat at the endpoint, it would be better if a threat could be blocked even before the potential threat was delivered to the endpoint itself. 

Blocking threats before they get to the endpoint

Reversing the clock by a few seconds, you potentially have the opportunity to detect or block a threat on the network before it even arrives at the target host. Network-based threat detection has the benefit of being able to throw more resources at the detection problem, as it is not trying to share resources with the rest of the endpoint operating system. As such, these solutions can sometimes detect threats that would take be impractical to detect on the endpoint. However, network protection systems have the same limitation that endpoints do:  they must be able to identify a threat within milliseconds in order to be able to block a threat without noticeably decreasing performance. Assuming that the network transaction must occur prior to the threat’s successful delivery and execution on the target, network enforcement solutions operate in the window before n – 100ms. A network detection solution can differ from a network protection solution is that, if blocking is not a requirement, the system can continue to analyze a threat after it has been delivered to the target host, and warn about the threat later, once it has identified it. This means that an IDS solution operates in the realm after n + 10 seconds or so. This is the basic difference between an Intrusion Detection System (IDS) and an Intrusion Prevention System (IPS). An IDS can only identify threats, whereas an IPS will also try and block threats that they can detect before the network connection is complete. Unfortunately, some advanced persistent threats can take minutes (or longer) to identify and cannot be blocked in real-time.

As network protection or detection solutions can protect resources regardless of whether they are protected by and endpoint solution, a combination of Network and Endpoint solutions can provide the best opportunity to block, or at least identify threats. That said, even the best detection tools will ultimately miss more recent or advanced threats. 

Preventing catastrophe with log analysis

Advanced threats can evade even the best endpoint or network security solution. Log analysis is often your last line of defense. Many security solutions provide some logging abilities, and when logs are aggregated into a SIEM, correlations between events captured in these logs can help identify alerts missed by other detection mechanisms.  Sometimes events can appear benign, but in context of other detections, can indicate that a breach has spread beyond the initially anticipated scope. 

The benefits and drawbacks of log analysis are one in the same.  Each log entry represents a different minor event that occurred. This may be an event on the network, an even on a server, an event on an endpoint, or even an event by an individual application or service. For the log entry to exist, the action had to have taken place already. In this circumstance, the log entry, at minimum, is created at n + 1ms. The log must then be collected and aggregated at the SIEM. For the sake of this exercise, we will say that log entries are collected hourly. So, the log entry makes it to the SIEM at n + 60 minutes. Depending on the complexity of the event, the SIEM may identify it immediately, or perhaps only when the event is correlated with another event. This time period is variable, so the even may live for hours, days or weeks before its relevance is noticed. 

In short, log analysis, by definition, can only provide forensic detection, as the important information always comes “post-breach.” Despite this, this detection is vital, as it is the only source of date regarding threats that have bypassed or evaded all other detection.

Just in “time”

Network and endpoint solutions operate closely around the “compromise event,” and each will impact “time-to-detection.” Depending on the complexity of the threat, time-to-detection may be in advance of the compromising event, or some point following the event. Log analysis frequently depends on second-order factors related to the event, so the time-to-detection is longer, but is also the only way to identify threats that have evaded traditional network and endpoint detection systems. As log analysis can often show the complex interconnections between compromised devices, it also helps decrease time-to-containment by allowing users to truly scope the compromise. 

In an ideal world, all threats would be identified in advance of a compromise or breach. In reality, threats are becoming too advanced to be detecting in the time period necessary to block them before delivery, if at all. When you take the time to consider the alternatives, you realize the need for a strategy to prevent threats, as well as identify the threats your solutions have missed.

There are a number of strategies for securing your network, and all work within a different window of time.

This article is published as part of the IDG Contributor Network. Want to Join?

SUBSCRIBE! Get the best of CSO delivered to your email inbox.