REVIEW: Threat Intelligence could turn the tide against cybercriminals

In recent reviews, we looked at the advancements in endpoint security, including new ways companies are employing technology like virtual machines to get a leg up on potential attackers. But despite impressive new defensive technologies, the bad guys still seem to be getting through.

According to security engineers we’ve talked with, the problem with network defense these days is two-fold. First, no matter how innovative the defensive technology deployed, it will eventually be breached or circumvented. And because most of the top attackers and groups collaborate, the tools and techniques used to successfully break down defenses are quickly shared.

+ Also on Network World: 7 free Wi-Fi stumbling and surveying tools +

On the other hand, most companies and governments have not traditionally shared data about successful attacks. So even if one company spots a vulnerability and fixes it, other companies can remain in the dark and be hit with an attack that could have been prevented.

Secondly, now that many security tools are generating alerts about possible threats, it’s almost like a dam has broken. There is literally so much random threat data circulating that it becomes difficult, if not impossible, for any one person or even one organization to parse it all out and find the relevant nuggets that relate to their specific situation.

That is where Threat Intelligence comes into play. This is a relatively new concept that is still being defined and modified by the very companies that offer it, even as it becomes a cornerstone of many network defensive plans. We got some hands-on training and testing with threat intelligence platforms from ThreatConnect, ThreatStream, Soltra, Arbor Networks and iSIGHT.

In general, there are two main types of threat intelligence vendors. First is the threat intelligence provider that finds external data about threats and emerging attack trends in order to share that data to subscribers. Secondly, some companies have built software platforms that pull in multiple feeds from potentially hundreds of sources and then sorts that data so that the most relevant threats are shown to users in the form of alerts.

But there is a lot of crossover. A threat intelligence provider may parse data for subscribers by industry or type, or even specific servers and programs running on a client network, while a platform vendor might provide their own threat feed streams in addition to just tracking others. Finally, many products in both groups offer some form of collaboration, making it easier for companies and organizations to share security information, while protecting any proprietary data from slipping out to competitors. Each of the threat intelligence products we looked at approached the topic differently, with each adding unique value to an organization’s security posture.

Here are the individual reviews:

ThreatStream OPTIC

One of the most advanced Threat Intelligence Platforms (TIP) we looked at, ThreatStream OPTIC is designed to process, analyze and rank threat data from more than 170 open source feeds, up to 30 or more commercial feeds and several more produced by government organizations. Data tied to threats that specifically endangers a protected network is then given to appropriate personnel.

ThreatStream OPTIC is designed to work in conjunction with SIEM tools like QRadar and Splunk to determine if the data from outside threat streams is of concern to protected networks, such as if any outgoing traffic is hitting known malware sites. Depending on the program that OPTIC is paired with, patches or remediation actions can be deployed or even automated.

However, if an organization is not using a commercial SIEM product, they can still use ThreatStream OPTIC because it integrates with the open source alternatives, something ThreatStream can setup for customers if needed.

+ ALSO ON NETWORK WORLD Threat intelligence needs to grow up +

The ThreatStream program is designed to be deployed behind enterprise firewalls so that all the matching of internal threats and data processing takes place internally and is never vulnerable to data sniffing type attacks. Nobody on the outside would have any way of knowing what OPTIC is doing or what data is being parsed. OPTIC itself is a relatively small file in terms of installation size and can be deployed on a single Linux virtual machine.

The amount of threat data that OPTIC has access to is impressive, though the real magic is how the program examines all of that data to find relevant threat information based on the specific network it’s protecting. It can even monitor some of the dark web channels used by hackers to see if, for example, any credentials stolen from a protected organization are up for sale, and then alert affected users to immediately change their passwords.

Another unique feature is the inclusion of the Modern Honey Network (MHN) platform as a potential threat feed. MHN is an open source honeypot deployment program that allows organizations to set up traps to catch malware that is targeting specific data, sectors or technology. Users can deploy as many honeypots as they want and feed the captured data into the system as a separate feed, or tap into existing nets that are already active.

Once a threat is identified by the feeds and matched to some internal network indicator, users can drill down and get information on what that threat was attempting to do, which can then be matched to known threat data on adversaries, tools and techniques.

Because information about threats is saved from the streams, researching a specific URL, for example, can be done anonymously because the analyst is looking at the threat data collected by OPTIC through the streams and not on the live Web. That way nothing like an IP address from a company security officer visiting a suspect site can tip off an attacker that their probing has been discovered. A further tool available in OPTIC is ThreatExplorer, which can help to visually show the links and connections between threats detected on the network with known threats streaming in from the global community and configured threat streams.

Once a threat is confirmed, administrators can share that data with their communities within OPTIC. Collected threat data can be carefully shared, and more or less information can be shared based on levels of trust established by the program. For example, sharing something publicly with all OPTIC users might use the least amount of data while sharing within a trusted circle of partners might include things like IP addresses or target data. That way, sharing is enabled for the good of the community without compromising any proprietary data, or anything that might inadvertently help the attackers.

ThreatStream OPTIC, which starts at $50,000, is a very advanced program that can make sense of a nearly unlimited number of threat streams, and then share intelligence within a select community of users.

ThreatConnect 3.0

ThreatConnect 3.0 is a Threat Intelligence Platform (TIP) that puts a heavy emphasis on collaboration and community. It’s one of the strongest platforms for those who believe that the key to winning the war against adversaries is to rally the affected communities to band together for mutual support and defense.

At the time of our testing, there were more than 4,000 active users on the ThreatConnect platform. A user does not necessarily mean an individual person, but could also represent an organization or an entire enterprise. ThreatConnect collects threat streams from multiple sources and then allows specific communities of users to collaborate on what steps work, who the adversaries are and what they are targeting. Users are even able to write specific apps that can be deployed through ThreatConnect, after being approved by administrators, to take actions that benefit the community, such as deploying a patch to a specific type of firewall to help block an emerging threat in a specific industry.

ThreatConnect can be deployed as a public cloud application, a private cloud application or as an on-premise solution. Company officials say it takes about one to two weeks to install ThreatConnect, make it the hub of security operations for an organization and train users. It might take slightly longer for an on-premise installation. Our test used the public cloud version.

Users of ThreatConnect are first evaluated based on where they sit on a five-tier security maturity model, with the goal of eventually getting every organization up to the final step in the model. At level one, organizations may be purchasing outside threat streams but not doing much with them. Level two is where they begin to process their own data, which might mean cutting and pasting log files into spreadsheets to look for threats and trends. Level three is when a company starts to incorporate threat data from others to compare it with their own, and where many customers begin in the maturity model with ThreatConnect. At level four, everything begins to get integrated, where alerts from internal Security Information and Event Management (SIEM) software is compared to external threat data from the streams to generate real threat intelligence. Finally, at level five, most of the internal security problems have been addressed and the organization can begin sharing its own collected data with the community, protecting not only their supply chain but possibility their entire industry and sector.

The main ThreatConnect interface is a splash page showing general information about the current state of threats, threat actors, victims and other indicators being tracked by the program worldwide, or by the specific communities that users join. To join a community, a user needs to apply. So the owner of a store might join the Retail Community while a bank might join the Global Financial Services community. Communities are administered and moderated by users, and individual access and membership must first be approved, so that only companies that are actually part of a community can have a hand in defending it. Once a user joins a community, the main splash page can be configured to show just that information.

Regardless of what communities are joined, from the main page each individual threat intelligence feed that an organization has access to can be clicked on. The entire interface is a drill-down model, where users can keep clicking for increasingly specific information about threats including IP addresses used for attacks, information about the threat actors, the MD5 hash of the malware being used and any contributed insights, documents or solutions offered up by the community. Once singled out, individual adversaries can be tracked so that new attacks that use the same techniques, servers or information can be linked back to the original threat actor – thus giving insight to their motivations and attack patterns.

One of the big advantages of ThreatConnect is the ability to input unstructured data. We were able to take a Threat Report PDF from a known anti-malware vendor and have the system scan it for things like the IP addresses being used by attackers in the report. That data then could be automatically compared to the existing threat data to see if any known adversaries working within an organization’s community are possibly involved with this new technique. You can also take that captured data and link it back to the original document, which can also be added to the system.

ThreatConnect works great on its own as a community defense platform for generating specific threat intelligence and making sense of all the available data. However, it can also be integrated with third-party programs to provide automation where specific threats found by the community can be automatically patched. That was outside of the scope of this review, but again, even without that component, ThreatConnect provides a very powerful collaboration tool that can put organizations on equal footing with adversaries while improving their cybersecurity maturity.

ThreatConnect offers a free edition as well as three paid editions starting as low as $45,000. The number of features, functionality and the chosen deployment model (private cloud, public cloud, or on-premises) determine the price for each edition.

Arbor Networks Pravail Security Analytics

Pravail Security Analytics is one of the easiest threat intelligence systems to use. Built by Arbor Networks, it’s also unique in that it does not provide alerts to users because company officials say that most analysts are in a state of constant over-alert fatigue anyway. Instead, Pravail is a tool designed to allow analysts to go hunting for threats and even to create rulesets that lets them play hunches and prove theories they can develop by observing the data.

+ ALSO ON NETWORK WORLD Arbor’s Fort Disco botnet has a hit list of more than 400,000 domains +

Pravail is installed as a two-tier appliance. The first component is the controller which collates data and stores it in up to a 64 terabyte drive enclosure for historical analysis. Each controller manages up to four collector devices depending on the size of the network being protected. Finally, collectors monitor multiple capture points, which are simply areas within the network where things like traffic and behavior are recorded. Deploying the capture points is strategically done in order to get the best results, which is one of the services that Arbor Networks provide. For example, a great place to deploy a capture point is directly behind the last firewall in a network, which would show any threats that somehow bypass all other protection. Other good points include switches and routers inside the network.

Because the capture points are set up internally, Pravail isn’t tied into being a perimeter security solution. In fact, Pravail did a great job at detecting the lateral movement associated with malware that has already gotten a foothold inside a network, and the telltale signs left by insider threats or disgruntled users as they attempt to disrupt or steal data from inside the perimeter. In addition to the main network capture points, Pravail incorporates two threat feeds created by the Arbor Networks research group and a third public feed. Users can easily incorporate any other feeds that they want to be a part of the platform or which they already pay to receive.

Pravail is designed to be simple enough that a junior analyst with a basic understanding of cybersecurity can immediately get some benefit from the program and master it in a couple days. But it also contains more than enough information so that a veteran professional will find a lot of value when using the tool. As such, the first interface that you see when you first log on to Pravail is more or less the jumping off point for the entire program.

At a glance, users can see the number and types of attacks hitting network capture points over time. Interestingly, the designers chose not to go with a traditional green, yellow and red model for the graphics because of the perception that analysts would only focus on the red colors and ignore dangerous threats listed as yellow or even green. Regardless of color scheme, hovering over a specific period of time will show the various severities of attacks hitting capture points. Data can be examined in real-time, though the historical analysis is probably the most interesting feature.

Looking at the threat data over time allows analysts to use their deductive skills and training to spot potential threats that would never be possible if all the data were not represented visually. For example, on the demo system, it was obvious that attacks spiked one Friday afternoon and then again on a Friday a few weeks later. Drilling down to look at the individual days is done by simply scrolling the mouse wheel forward. The data gets increasingly granular until the viewing of second to second activity is possible.

Once a threat is located and highlighted, all the known indicators associated with that threat can be viewed. In the case of the odd activity in the test system, it was quickly discovered that the initial burst of attacks and the latter group were related because they were using IP addresses that were registered under the same phone number, which indicates that they likely came from the same attacker. It was determined that the first group of low-priority attacks were simply scanning and probing. The latter group of more severe attacks happened because the same attacker was using an exploit they initially discovered against a single network client.

Once all the techniques were recorded and added to the available data and associated with the attacker, the data could be pivoted, turning the exploited machine into an attacker in the Pravail interface instead of a victim in the system. That way, it was easy to view any attacks that were launched from the compromised machine outside of the network, and also laterally within the perimeter. In a very real sense, Pravail gives analysts the tools they need to become detectives again, and to deploy their skills and knowledge against attackers instead of just reacting to bells and whistles.

The ability of Pravail to store massive amounts of log and event files comes into play once again because of the ability to reanalyze old data, a process the software calls looping. Because zero-day attacks might not get caught at first, and may not show up at all until new patches or tools arrive, looping allows administrators to use Pravail to scan historical data from weeks or months ago using whatever new tools are available. That way it can be determined not only if a network is safe in the present, but if it was breached before using a zero day or similar exploit that was undetectable at the time. If a breach did occur, then a normal investigation can follow to see who the adversary was and what data they tried to modify or steal.

Pravail is likely one of the most useful threat intelligence tools for folks that want to take an active role in network defense. Any organization with good analysts who are not doing much other than reacting to threat alerts should consider Pravail. Pricing starts at $75,000.

iSIGHT Partners ThreatScape

iSIGHT Partners was there at the beginning of modern threat intelligence when it formed in late 2006. Its primary role is that of a threat intelligence feed provider, though it also has its own platform called ThreatScape, as well as a browser extension for entry level threat intelligence gathering. Most of the platform provider companies in this review touted their ability to use the iSIGHT feed as a feature of their program, a testimony to how respected it is within the threat intelligence community.

The reason the feed is so good is because iSIGHT deploys over 200 operatives around the world whose full time jobs are collecting threat data and turning it into not only a proactive model but also a predictive one that can anticipate future threats. Analysts fill various roles in that chain that include virtual spies who operate within global hacker and criminal communities which often communicate within the dark web. The spies collect information about the bad guy’s plans, tactics and ambitions. Other iSIGHT analysts combine that human intelligence with machine intelligence generated through various traps and feeds. Finally, a large group takes all of that information and uses it to keep the ThreatScape tools updated and the community informed about pending and existing threats.

The iSIGHT data can be delivered in a variety of ways to subscribers, and also can feed directly into most threat intelligence platforms produced by other companies. There is a daily e-mail at the least interactive level, though it can also be helpful, especially for those companies that have not created very advanced threat intelligence platforms. The end of each e-mail contains threat indicators which can be easily cut and pasted into whatever defensive platform is being used, or just into a spreadsheet if that data is still being collected and stored by hand.

Secondly, there is an online portal that contains a huge repository with every single piece of threat intelligence the company has ever produced going all the way back to its founding. And there is a software developer’s kit so that apps can be created to help improve security in various communities.

One of the most interesting and unique methods of getting to the iSIGHT feeds is through a browser plugin. It currently only works with Chrome, though there may be plans to expand to other browsers in the future. Using the browser interface is incredibly simple. From the normal Chrome screen, users can scan web pages to search for any indicators of malicious activity. From there, queries can be sent into the main program API to check those indicators against specific campaigns.

It’s also quite easy to pivot into the iSIGHT web portal so that any hint of malicious activity that has ever occurred can be located. For example, we were able to use it to check on specific campaigns, such as the recent #OpBaltimore attacks targeting that city’s cyber infrastructure in conjunction with physical protests.

As an entry-level portal for a company that has not yet jumped into threat intelligence, a quick access point for a security officer needing to hurriedly check something or as a nice addition to the main ThreatScape API, the Chrome browser extension is helpful and easy to use. And it stays out of the way of normal Web-browsing activity, only activating when requested.

The main program also has a pleasing interface. It’s not as easy to use as the Web browser, but has a lot more features akin to a typical threat intelligence platform. Besides importing and searching for indicators, integration with third-party programs enables ThreatScape to automatically take certain defensive actions if enough threat indicators are present.

In addition to providing one of the best threat intelligence feeds, iSIGHT also offers to train companies how to best use that information. Called the Intelligence Integration Services (IIS), it provides one-on-one training on how to make use of threat intelligence. With threat intelligence being such a new and constantly evolving concept, that level of assistance is likely to be highly appreciated by customers who are unsure how to proceed. In fact, it’s probably one of the biggest missing components in threat intelligence; showing users how they can turn all that collected data into actionable intelligence, and then how to actually do something about it to protect their networks from specific threats.

Enterprise subscriptions are offered on a tiered basis for each of six distinct intelligence offerings branded as ThreatScape. Depending on deployment configuration, prices start at $75,000 for one ThreatScape.

Soltra Edge

Soltra Edge isn’t so much a threat intelligence platform as it is a threat intelligence protocol that happens to also work as a platform. Soltra is owned by two non-profit companies, the Financial Services Information Sharing and Analysis Center (FS-ISAC) and The Depository Trust & Clearing Corporation (DTCC). As such, the purpose of Soltra Edge is not to make money, but instead to make things more secure, especially for financial companies that both groups serve. However, anyone can benefit from Soltra Edge. In fact, it’s the only platform in this feature that can be downloaded and used by anyone for free regardless of whether they are an individual, an organization or anything in between.

The idea behind Soltra is that one of the biggest problems within threat intelligence is that not only do companies have trouble getting the concept of sharing threat data into their culture, but in many cases the proprietary format of the threat feeds prevents it. Soltra fixes this problem by backing the universally accepted open source Structured Threat Intelligence eXpression (STIX) language that machines can use to communicate threat data to one another. It also taps into the Trusted Automated eXchange of Indicator Information (TAXII) transport protocol to safely and easily move the STIX threat data around, kind of like how TCP/IP moves data packets to and from any network. Many threat stream vendors are starting to support STIX and TAXII, though many resist for various reasons. Soltra however has translation apps that can convert most proprietary feeds into the open STIX and TAXII format.

One of the biggest advantages of using Edge, beyond the price, is the fact that any formats that are already expressed in STIX can automatically go into the platform regardless of their delivery method. So users don’t have to manually cut and paste data from feeds to quickly build up a large dataset which can be examined for threats.

Even unstructured data can be added to Edge. This is a more manual process, however the program does a great job of asking the user various questions in a certain order so that a STIX record can be created, even if the user knows nothing about STIX. It does all the heavy lifting on the backend.

Edge is deployed and runs as a virtual machine on any system that supports the Oracle Database Appliance format. There is a console interface that lets users spin up various services, but once actually running, the front-end interface is not dissimilar from other threat intelligence platforms. You can call in external feeds that support STIX and TAXII, and there are quite a lot of them, or create your own repository of internal system data.

It is a bit more work to use Edge than many of the other commercial threat intelligence platforms, and the interface is not quite as pretty, though some may prefer the more uncluttered layout. Data about threats stream into the system automatically using the TAXII protocol and can be immediately analyzed and shared with the rest of the community that also has their data formatted into STIX. There are no native communities by default inside Edge, but given that everyone using the platform has the exact same format and transport protocols, setting one up is a breeze once all parties give permission, and is automatic thereafter.

Edge, through its use of STIX and TAXII, has the capability to change the whole threat intelligence landscape even as the companies operating inside it continue to define the term. If it ever reaches a tipping point, then an actual threat intelligence network could be born that automatically shares its data on threats at machine speed with members. And that club is far from exclusive, with everyone invited in and not much of a cover charge. The Edge platform itself probably isn’t quite there yet where such a dream could be realized, but it’s on the right track, has a solid base of operations, and continues to develop.

John Breeden is an award winning reviewer and public speaker with over 20 years of experience. He is currently the CEO of the Tech Writers Bureau, a group of influential journalists and writers who work in government and other circles. He can be reached atjbreeden@techwritersbureau.com.