Recent results of the first-ever MITRE Engenuity ATT&CK Evaluation of security service providers give potential buyers a deeper look into the capabilities of 15 security services providers, and sheds some light on their skills in detecting, analyzing, and describing adversary behavior.
MITRE Engenuity first introduced an ATT&CK Evaluation for managed services five years ago and has since then conducted many independent evaluations of cybersecurity products using an open methodology based on the ATT&CK knowledge base. The evaluations use simulated real-world attacks to evaluate the tactics, techniques, and procedures (TTPs) of relevant advanced persistent threats (APTs). In addition, the vendors must demonstrate their ability to detect, analyze, and describe those activities.
MITRE’s goal is to help both cybersecurity vendors and their potential customers make informed decisions to combat cyberthreats and improve threat detection capabilities.
“Most buyers want to understand how a vendor is going to respond to a threat after they've detected it,” says Eric Kokonas, Global Head of Analyst Relations with Sophos. “So, it's a fantastic evaluation MITRE is doing. It’s filling a huge gap that's existed in the marketplace and provides a tangible way of evaluating vendors.”
In one example, the assessment tested vendors’ abilities to detect and analyze attack tactics and techniques simulating those used by OilRig, an Iranian government-affiliated threat actor – also known as APT34 and Helix Kitten.
“The evaluation was closed book and it was on the vendors to determine who the testers were emulating as part of the attack,” says Kokonas. It was a detection-only evaluation and MITRE Engenuity did not evaluate vendors’ ability to act on and respond to threats.
What To Do with Vendor Results
The evaluation is not intended to be a stack ranking of vendors, he notes. No vendors “win” the evaluation and no one is considered to have performed the best, he explains, Instead, it’s designed to reveal what the relationship with an MDR partner is going to look like should an organization choose to work with them.
“The results help organizations look at examples of how vendors communicate with customers,” says Kokonas. “For example, is it automated? Is it clear, actionable information? What is the quality of content?”
Sophos MDR successfully reported malicious activity across all 10 MITRE ATT&CK steps, excelling in its ability to detect and respond to sophisticated threat actors with speed and precision. The results allow organizations to evaluate Sophos and the others who took part in the exercise clearly to ensure trust and alignment in selecting an MDR provider.
“It’s an evaluation of a vendor’s detection capabilities,” says Kokonas. “Potential customers can understand what response actions they can take, what collaboration with the provider looks like, and other important factors for determining how working with them might be like.”
For more details about the MITRE evaluations and their results, visit https://attackevals.mitre-engenuity.org/managed-services/managed-services.
To learn more about Sophos MDR services visit Sophos today.