Do you really know what happened during that data breach?

Being in the Endpoint Detection and Response business (full disclosure: I manage the development of the CyFIR Enterprise Forensics and Incident Response product suite), I’ve noticed both an interesting and disturbing trend from all manner of cyber security vendors that CISOs—especially those who didn’t come specifically out of a digital forensics discipline—should understand when listening to sales pitches from people like me.

Given the history of the profession, the majority of digital forensic practitioners came out of law enforcement. Before “Incident Response” was a buzzword included on every IT professional’s resume, most digital forensics casework supported the investigation and prosecution of crimes. To do so, digital forensic examiners would pore over a hard disk, often with rudimentary, low-level tools for days, weeks, or even months. Our tools could see every file on the hard disk, even those that the bad guy thought he had deleted, hidden, or locked away. We would scour disks looking for hidden partitions, host protected areas, or other spots that craftier suspects would use to hide their data from Johnny Law. When we found files of note, we would render them down to hash values—a digital “fingerprint” as it’s often called—to prove their uniqueness or to track their movements between systems or individuals. We could recover fragments of files deleted long ago from the hard disk, often finding a crucial piece of evidence. Forensics is an often difficult, time-consuming set of processes that can yield unimaginable results—if you’re willing to put in the time and effort.

That said, today’s EDR marketing landscape is trying to push purchasers into the belief that “AI will save us all” or that “machine learning keeps your network safe.” Truth be told, some of the newer “next-gen” antivirus tools really are impressive, even if they often don’t work exactly as they’re pitched. However, as forensic tool vendors like me want to sell the myth of the “find all evidence button,” these companies are telling weary, underfunded CISOs that their tools not only will stop myriad attacks, but also that they offer a “forensics component” on the off chance that something evil should get through their defenses.

If you’re looking at one of the ever-present meatball charts that compare different vendors’ tools against each other, you’ll often find that antivirus, patch management, continuous monitoring, or other programs under the EDR heading will have a proud black dot in a row called “Forensics” (often from an up-charged component). As a CISO, you can purchase one of these tools and check-off “Allows a user to perform a digital examination on a computer or network” from your readiness list, right?

I wish it were that simple.

When evaluating the “forensic capability” of a cyber security product, you need to ask the vendor some direct, pointed questions to learn what that specific vendor defines forensics to be. Finding and deleting the offending file is only part of the job; understanding the attack vector, reviewing the data exfiltrated, and quantifying the damage done are equally important in handling a breach and in preventing future attacks. Without knowing what went wrong, how can you be sure that you’ve taken the appropriate measures to stop it from happening again?

Question: Can an authorized member of my security team navigate to the hard disk structure on an endpoint to look at the content of individual files?

Why you care: Attacks often leave behind forensic evidence that is critical in the discovery of the type and amount of data that has potentially been exfiltrated from your organization. If you can’t find and view the content of the exfiltration files, you might not have accurate information regarding the size or scope of a breach.

Question: Can I pull running processes individually out of memory for external review, or at a minimum can I use your tool to extract the live RAM remotely for the entire machine?

Why you care: Strategic or advanced attacks may use custom-crafted malware that might be able to defend itself from antivirus engines or even automated sandboxes. Sometimes a manual breakdown of a malicious program’s capabilities is the only way to know the potential extent of any damage it caused, and to do that, you must be able to isolate and extract the process from live memory.

Question: How many endpoints can I search at once now that I know what I’m looking for?

Why You Care: Many tools that search remote endpoints are limited to searching only a few at a time through a round-robin scripting method. If you have a lot of time and money, that’s fine. If you’re short on either, look for tools where searching the endpoints happens simultaneously instead of five or ten at a time.

Question: Can I look through the raw data on the hard disk remotely and recover deleted files?

Why you care: Deleted malware, erased exfiltration files, and other items hidden from normal view of the operating system can provide critical evidence as to the scale and effectiveness of a breach. Without the capability to directly access a disk and recover deleted information, you’re likely to miss the whole picture. If you can’t do it remotely, you’re going to pay your employees (or a contractor) a lot of money to visit your individual locations and make copies of hard disks for later analysis.

Question: Can your solution help me from attackers who are ‘living off the land?’ or using fileless attacks?

Why You Care: Many platforms sold under the EDR banner are strongly based in their antivirus or continuous monitoring roots. While they may flag malicious activity in the form of a trojan or virus, they might miss the use of legitimate administrative tools by a bad actor. As a use case, ask how the solution being presented can help identify someone doing bad things by using stolen legitimate credentials and standard administration tools, and more importantly, make them show you.

Once you start digging with questions like these (and making the sales engineer pitching the product a little uncomfortable), you’ll find that the term “forensics” is being redefined by each individual software vendor for their own convenience—and a tick-mark on that meatball chart. If you have any doubt, find an old-school cop who has been doing digital forensics for twenty years and ask them if they consider “Tool X” to be forensically sound. Believe me, they’ll know the difference.