Can we really automate how security analysts think?

Through a combination of human and machine intelligence, security will become smarter, faster, and more effective. It can't come soon enough.

5 api automated robot keyboard
Thinkstock

In some conversations with security leaders, I inevitably run into a skeptic view that automation will never be able to replicate the decision making of security analysts. The truth is we can already automate a lot more of the decision making today than was possible just a couple of years ago.

Hence, it is imperative for security practitioners to understand the full capabilities and limits of modern automation technology. How will it help? Which SOC tasks can really be automated? And how the does the role of security analyst change as daily operations in the SOC become more automated?

We’re at the beginning of a long journey

Let me say upfront: I don’t think automation will completely replace the need for security analysts. Best way to approach automation is to view it as a force multiplier, i.e., how can you improve an analyst’s productivity tenfold with automation.

Security analysts don’t need to be spending as much time as they currently do on rote and mundane matters. By handling repetitive and low-complexity tasks, security automation can free analysts to tackle more complex work that requires sophisticated analysis such as event correlation.

Security automation is already proving it can help with complex analysis in the SOC. In fact, we’ve seen that by applying intelligent automation, it’s possible to have machines eliminate 95 percent of the false positive security alerts overwhelming security analysts today, freeing those analysts to spend more time on threat mitigation and proactive threat hunting.

Even with impressive results, the work of security automation is far from finished. In fact, I believe we’re only at the beginning of a long process of automation and optimization that will probably take 10 to 15 years to complete.

That might sound like a long time, but consider how long it has taken automation to transform the design of automobiles. From engines that had to be manually cranked to start to electronic ignitions; from manual transmissions to automatic transmissions; from cars that had to be steered by humans to cars that can park themselves; and a growing number of models that can actually navigate city streets. Automation has made driving faster, easier, and safer, but the transition to our current state of the art, with fleets of self-driving cars likely available within a few years, has taken many decades. In comparison, my prediction for 15 years for SOC automation seems like the blink of an eye.

Bringing automation to the SOC

What’s the first step on this journey?

Today most security analysts would be comfortable with automating the creation of SIEM rules and routine tasks to mitigate threats associated with those rules. For example, let’s say a threat intelligence feed warns of a certain type of traffic on a certain port. A SIEM rule can be created to monitor that port for the suspicious traffic. A threat mitigation task might be to block a particular source IP address if the suspicious traffic is detected.

This basic type of automation is found today in a growing number of security automation products. This example illustrates the fundamental nature of automation. There’s a pattern, which in a SOC could be a rule or an event, and based on that pattern, the automated system will take an action or make a prediction. That prediction in turn will lead to specific outcomes, such as which SOC playbook to follow to mitigate a threat.

An SOC automation solution needs to capture these patterns and store them in a repository, so that security intelligence grows over time and the automated solution can act on an ever-broadening scope of rules, events, and threats.

What should be automated?

Generally speaking, every possible task or feat of analysis that can be automated with confidence should be automated. Meanwhile, human analysts should monitor, guide, correct, and fine-tune security automation systems to train them for accuracy and comprehensiveness.

Why try to automate as many SOC tasks as possible? Humans, when overwhelmed, tend to take short cuts and skip steps, not to mention burn out by having to do rote repetitive tasks day in and out. In cases like this, it’s helpful to have a machine rigorously follow the playbook or script developed by analysts to mitigate an attack and restore data security.

Another reason is speed. Let’s say a SOC playbook calls for 30 different steps to be taken sequentially to mitigate a specific type of threat. A security automation solution can perform those tasks faster than a human analyst can, since the automated solution doesn’t have to type, walk from one workstation to another, or perform other physical tasks.

SOCs need this automation. Many SOCs have developed thorough playbooks, but when security events occur, perhaps only 20 percent of SOC analysts follow every step their playbooks describe. Instead, fatigue or pressing deadlines lead to many steps being skipped or overlooked. As a result, even the best knowledge available now is only irregularly applied to threat mitigation. Security automation can improve this performance, helping analysts ensure their own best thinking is applied more quickly and consistently to mitigate threats.

Yes, machines can ‘think’

Perhaps the great value that automation can provide is cognitive analysis: machine learning that applies event correlation and other advanced techniques to identify threats, including Zero Day, threats more quickly and accurately than ever before. Machine learning is already performing similar advanced feats in other fields, such as oncology, recognizing patterns more quickly and accurately than human experts. There’s every reason to believe similar machine learning techniques can be applied to IT security as well.

Ultimately, ROI will determine the pace and scope of automation. If automation improves the SOC’s effectiveness for an affordable price, it will likely be broadly adopted.

Looking ahead to a busy future for security analysts

Automating scripts to close ports and add IP addresses to blacklists sounds straightforward enough. Can security automation really advance to the point where it is performing advanced cognitive work? And if it does, will security analysts still have jobs? Yes and yes!

For comparison, it might be helpful to consider another high-risk field that has been transformed by the use of automation: aviation. If you told pilots in 1965 that someday autopilot systems would be able to handle much of the work of flying large aircraft on long, cross-country trips, they would have said you’re crazy. Aircrafts might have progressed from propellers to jet engines, but the business of flying itself was a hands-on job requiring advanced, highly trained personnel.

No, you might insist. It really will be possible for autopilot systems to perform much of the cognitive work done by pilots. “All right,” the Mad Men-era pilots might reply. But only in ideal conditions: fair weather, no wind. And yet today, many commercial aircraft are required to rely on autopilot to land when weather conditions are poor, such as when clouds are below 200 feet or when visibility is less than a quarter mile. In other words, airlines trust machines more than people to handle the most difficult, risk-fraught situations, such as landing in fog.

And yet pilots still have jobs, in fact there is a pilot shortage. Autopilot technology handles all the common situations, leaving pilots free to concentrate on any anomalies or risks, to which they can now devote their full attention. And human pilots still perform most take-offs and landings, especially those with good visibility.

A similar division of labor could develop in the SOC. Security automation solutions could handle routine matters and aid with security analysis. When crises occur, security automation solutions could take action quickly and accurately to secure the enterprise. Meanwhile, analysts would be free to perform their advanced analysis and make their own deductions, which eventually will be applied to the SOC playbook that drives the activities of both SOC applications and security analysts.

Through this combination of human and machine intelligence, security will become smarter, faster, and more effective. From the vantage of SOCs today, where analysts are facing advanced persistent threats and data breaches are rising 40 percent year-over-year, smarter, faster and more effective future can’t come soon enough.

This article is published as part of the IDG Contributor Network. Want to Join?

SUBSCRIBE! Get the best of CSO delivered to your email inbox.