Zeek: A free, powerful way to monitor networks, detect threats

It’s common in many organizations for network and security operations to exist in silos.  This may have been OK a couple of decades ago when threats only came in through a single point and applications were vertically integrated. Today, however, the environment is completely different, as the cloud, mobility, Internet of Things (IoT), and other trends have fundamentally changed the security landscape. Consequently, finding the source of a breach has never been more difficult.

Security professionals need network data

Today’s CISOs and security architects need to think differently and turn to the network for a source of information. The world has become network-centric, and the network holds a wealth of valuable information that’s relevant to security. The problem is legacy network monitoring tools are designed for network teams, which prevents security professionals from being able to tap into that network information and extract the information they need.

One solution to that problem is an open-source network monitoring platform called Zeek. It lets security teams see more, resulting in faster threat detection and incident response times.

Bro, now Zeek, turns network data into security intelligence

Zeek, known for the past 20 years as Bro, was developed in 1995 by Vern Paxson, a co-founder of Corelight. The project was initially named Bro as a reference to George Orwell’s Big Brother from the book 1984 as a reminder that monitoring goes hand in hand with privacy violations. The leadership team of the project realized Bro also had negative connotations and chose to rename it Zeek, which is a reference to the Far Side comic strip.

The platform is free to use and is available as open-source software, designed to analyze complex, high throughput networks. Zeek effectively sees everything because it extracts over 400 fields of data from network traffic in real time and across 35-plus protocols. These range from layer 3 to 7 and include HTTP, DNS, SSL and much more. 

The logs provide nearly the fidelity of full traffic packet capture at less than 1 percent of the file size, and logs are organized by protocol with fields extracted specifically for the security operations center (SOC) so they can make fast sense of the information.

Zeek is also a programming language

Zeek is also a programming language that enables users to write their own custom scripts to extract custom network data or automate monitoring or detection tasks related to network behaviors. These Zeek scripts can accomplish tasks such as identifying mismatched SSL certificates or the use of anomalous software or keyboards (e.g., an English language keyboard that changes to a Cyrillic keyboard). The fact that Zeek is both data and a set of tools to automate data insights makes it an especially powerful security platform.

Historically, security teams may have used a SIEM to combine network traffic logs with endpoint information, third-party intel feeds, and other sources of data. But the massive amount of data that SIEMs collect makes them difficult to work with, as the false positive rate is very high. Somewhere in the immense volume of information lurks malicious traffic, but locating it is like finding a needle in a haystack. Businesses have spent millions on technology to alert on potential problems, but each security alert often raises a series of new questions, such as:

• Is the alert real?
• When did it start?
• Where did it come from?
• How serious is the threat?
• How widespread is it?

For security professionals, network data is the source of truth

For most network and security professionals, network data is often viewed as the source of truth, as the network sees all. However, there are many types of network data, each of which provides a different level of information. NetFlow is often used, but that data is quite sparse (kind of like a cell phone bill — date, time, IP addresses, bytes sent/received, etc.), and it’s usually sampled, meaning captured periodically. This can leave huge gaps in visibility. It’s great for trending, but not useful for real-time forensics.

The antithesis of NetFlow is PCAP (a literal copy of the entire packet flow from a network), but that’s just raw data and not easy to work with or fast. A good analogy for PCAP is being handed a box of DVDs and being told to find a certain scene in a movie. It can be done, but it requires a lot of time and people resources. 

Zeek is a better source of data

Zeek was designed to be a better source of network data for threat hunting and incident response. Think of Zeek as an open-source security monitor that gives rich, organized, and easily searchable data to protect the environment without overwhelming network and security teams with useless information that can bog them down. Zeek extracts hundreds of fields of network data in real time and leads to faster incident response by providing fast and easy access to actionable information.

Also, the protocol has some useful features, such as a unique connection ID that lets the user see all protocol activity for a single TCP connection. It also time-stamps and synchronizes the data across the various log files. That makes threat hunting much faster. For example, without Zeek, investigating an IDS alert based on HTTP traffic would involve numerous steps, including retrieving NetFlow logs and PCAP information, querying PCAP, extracting files and then hashing them, observing file transfers, checking the file again, and hoping the source of the alert was found. With Zeek, the administer can pivot off the connection UID, see what file was transferred as the source of the alert, and then check other hosts in less than a minute. 

In the early days, the use of Zeek was concentrated in government intelligence agencies, defense groups, labs and major universities, but today it is used by thousands of organizations, including some of the world’s largest enterprises.

There’s no question that Zeek is a great tool, but like all open-source software, getting Zeek up and running can be a challenge. A server needs to be built or procured and then a cluster assembled. Zeek needs to be installed, the configuration customized and then be tuned to work with other security tools. Despite the value Zeek brings, some might not feel up to the task of getting it up and running. Open-source software certainly has its advantages but also has some downside.

Corelight simplifies the deployment and use of Zeek

As I mentioned before, Zeek was created by one of the founders of Corelight, which makes easy-to-deploy sensors that let businesses take advantage of Zeek in a matter of minutes, instead of the typical weeks or months it takes to configure and deploy the open-source version at scale. Corelight Sensors come in both virtual and physical form factors and support a range of analysis throughputs up to and beyond 25 Gbps; the sensors are plug-and-play solutions that have Zeek installed and ready to use. 

The installation and configuration process is simple via Corelight’s web console. The other benefit of using Corelight is that the company provides excellent support, as they are the group that created Zeek and maintains the open-source project. Also, integration becomes faster because the pre-loaded software includes automatic updates and integration with many mainstream security tools.

There is also support for custom scripting to create custom detections, as well as log filtering to tune the log volumes and manage SIEM and data storage costs. Corelight includes several helpful management features, such as sensor health and performance monitoring and APIs for automation.

There’s an axiom in networking and security that says, “You can’t secure or manage what you can’t see,” and the majority of security and network management tools give you only a small part or low-resolution version of the picture you need because they were not designed with security in mind. Zeek provides more depth and gives a full picture at the right resolution for the vast majority of security use cases. 

Zeek is open source, so it’s free to use, and there is a wealth of knowledge, support and scripts on Zeek.org. Many IT organizations will have the technical chops to build their own stack and leverage the benefits of the open source framework. Those that don’t can turn to Corelight as a way to get Zeek up and running quickly. In either case, Zeek is a platform that network and security pros should try if aren’t already running it.