How does a company operationalize its risk and security programs? More specifically, with all of the talk about big data, how does a company operationalize its threat intelligence process?Many companies think they know what the keys are to their kingdom and where the entry points are located. Unfortunately, they soon find out that the most serious breaches often take place somewhere else.+ ALSO ON NETWORK WORLD: 5 tips for better enterprise security +\u201cCompanies will watch their ATM activity and miss the subtle warning signs passing through their mainframe,\u201d says Sharon Vardi, Chief Marketing Officer at Securonix. \u201cWithout knowing it, companies are leaving their crown jewels sitting exposed, prime for the taking.\u201dKnowing what to watch requires collecting data that can be analyzed, and allocating eyes to perform the analysis.However, organizations won't succeed if they don\u2019t collect and analyze an ongoing full stream of data\u2014success requires more than just a snapshot from a limited window of time. The data needs to be collected before, during and after malicious activity takes place.\u201cCompanies also need to include data from inside and across the network, from each and every endpoint, and potentially even from external and public sources located outside of the network,\u201d says Alan Hall, Director of Strategy & Product Marketing for the Advanced Threat Protection Group at Blue Coat Systems. \u201cOtherwise, responses will be limited, at best.\u201dEffective incident response requires contextBeing able to respond to an incident is where the rubber meets the road. This requires context\u2014information beyond what is captured in raw form. Context can be used to identify an advanced or otherwise covert attack or compromise\u2014plus give you the means to determine how best to react.Security data isn\u2019t big data. It is morbidly-obese data.Travis Smith, Senior Security Research Engineer at Tripwire\u201cTo properly manage security incidents, organizations not only need to collect data but also analyze the data in real time\u2014and then store that data so it can be used later to correlate against new real-time data,\u201d says Travis Smith, Senior Security Research Engineer at Tripwire. \u201cThe challenge is\u2026storing data costs money\u2014plus, the management and usage of the data can be a real problem as well.\u201dThe reality is, security teams looking to analyze logs are at the mercy of the developers who decide what to log and from which systems. These details are often built into (or more accurately, excluded from) systems when they are developed.Full packet capture uncovers the real meatEven still, security logs are just the tip of the iceberg. The real meat is in full packet capture across the entire network. Getting past this log-only barrier and into network capture leaves companies with a load of security data\u2014yet another challenge: \u201cSecurity data isn\u2019t big data,\u201d says Smith. \u201cIt is morbidly-obese data.\u201dThe normal best practice for data storage is 30 days of traffic, though some industry policies require more and some government regulations demand more. \u201cIt\u2019s almost negligent if the security team is living purely in alert mode, unable to analyze context,\u201d adds Hall.Sometimes it\u2019s more than just a matter of how much, it could also be an issue of how: customers seem to be struggling to get what they want from their security management programs. \u201cSecurity teams either get no alerts or too few alerts...or they suffer from severe alert fatigue,\u201d says John Humphreys, vice president of marketing at Proficio.Other sources of data to considerAs Smith at Tripwire recommends, absolutely capture your log data but also look to move beyond logs and \u201corganize some of your own internal network feeds. You should also tie sessions together to capture packet strings and ultimately perform a full packet capture.\u201dVardi adds, \u201cYou should also consider external feeds of data that may not be traditionally categorized as security data.\u201d This includes Facebook activity, job searches and other data sources available to you while your employees are operating from your corporate-owned and corporate-liable devices and networks.\u201cOpen Source intelligence with company data is fair game in these circumstances,\u201d adds Vardi. These data sources may not look like security data but can dramatically change the context of the security data and provide a new way for companies to look at their risk profile. Of course, to make threat intelligence useful, the feeds must be credible and based on reliable sources. This includes your own internal feeds. There are a large number of apps that generate a ton of seemingly-non-malicious internal traffic, most of which are designed to share data so the business teams can do their work. Still, the inclusion of these data feeds and the quality of this data can\u2019t be overlooked.These internal-only network communications are often dismissed or go undetected by only monitoring the intrusion and exfiltration-detection system logs. This is usually because the traffic moves horizontally within the network and never crosses the intrusion monitoring systems nor hits the path of the perimeter firewall.\u201cIntrusion and exfiltration only happen when the device traffic enters or leaves the corporate network,\u201d says Carmine Clementelli, Sales and Marketing Manager at PFU Systems, a Fujitsu company. \u201cSimilarly the command-and-control communications happen off the network by using external, temporary websites. Most times, finding the problem at this layer means it\u2019s too late.\u201dWhat context to look for?When it comes to determining the context that will be used to seek out the threats an organization faces and the attacks they are experiencing, there are generally one of three options to consider:Let the systems define the context automatically and hope their vendor-defined configurations and rules \u201cget it right.\u201dUse your own learned context that you\u2019ve garnered over time and hope that you know enough about your environment\u2014or at least as much as the attackers know.Define the context in an on-the-fly, ad-hoc fashion; try to pull in the threat data and supporting intelligence to match; and then pray you can stay ahead of the game and not fall prey to alert fatigue.Or you can take advantage of the security community and use cross-industry, cross-profile sets defined by others to select and then customize the context. \u201cSecurity teams need to observe their IT life in reality by using other companies\u2019 experiences,\u201d says Humphreys. \u201cThis is a good way to understand the real context.\u201dWhen it comes to insiders stealing data and sending data to competitors, the context here lies in seeing your employees and contractors accessing data more frequently than is typical. You might also capture traffic that shows employees are sharing data outside the organization, such as via a personal email account or a removable USB drive.An employee who recently had a poor review could be flagged as an even higher risk. If a third-party vendor makes multiple login attempts and attempts to access company systems not typically accessed, it could be a sign that the vendor is either acting maliciously or was hit by a phishing attack.But it\u2019s not just people and systems that provide context. \u201cA document can be an entity as well,\u201d says Vardi. \u201cThe behavior of a document is equally important to watch. Where does it live? Who accesses it? From which IP addresses is it accessed? Where does it travel to?\u201dVardi adds, \u201cEach of these\u2014when viewed together along with other events and alerts\u2014can bring additional context to otherwise undetected malicious activity. For example, if an employee, partner or customer usually logs in from a Windows PC using Firefox, and all of a sudden they download documents from a Mac using Safari, then that could be a sign of trouble brewing.\u201dATM fraud is another real-world example that\u2019s becoming a big deal these days. Picture a banking customer who has been with the bank for 20 years and interacts with the bank a certain way most of the time. You can look for anomalies in their activity: the amount of their withdrawals, the location of the withdrawals, and the card used. Perhaps even the number of times the card is used during the day at different locations.You can use this same principle for monitoring access to business resources and other user and system activities on the network\u2014not just ATMs and withdrawals.Here are a few examples:An endpoint allocated to a single user logs onto the network multiple times from the same location using multiple user-identities. If you see this, there\u2019s a chance the system has been compromised.Unencrypted North\/South traffic correlated with internal East\/West traffic\u2014be on the lookout for network activity coming in and moving laterally. This connected flow could be the sign of an unauthorized user or device on the network.Leverage behavioral-based detection techniques that look at outbound traffic and peer-to-peer traffic to see where the traffic is going and how frequently it travels that path. Focusing on the ingress shouldn't be the only approach; you also need to assume the malware is already inside and monitor the egress.Take advantage of command-and-control detection as well to identify existing attacks that are looking to exfiltrate data. Be aware that oftentimes the data exfiltration doesn't come as a single download; it can happen as a series of small actions over a long period of time. What happens in the middle that represents the long period of activity are the lateral movements\u2014based on behavior as opposed to just packet analysis. Consider the use of an IT\/security approved website that\u2019s been hijacked by the attacker as a storage service that won\u2019t be detected by your reputation and filtering systems.Go beyond top-level application monitoring to analyze application features that are being being used. Facebook as a whole may be deemed OK for some employees, but how and when are they using Facebook chat vs. Facebook video view vs. Facebook video upload? What and how much data is being transferred to\/from each of these features?How will you respond?Context is not only necessary for detecting an attack, it is also paramount in identifying the source of the attack, blocking the spread of the attack, and fixing what\u2019s been compromised due to the attack.\u201cWith integrated detection investigation, analytics and forensics, you can see a zero-day alert on a network from four months ago,\u201d says John Dasher, vice president of marketing at Niara. \u201cYou can then look at your logs, packet flow and threat feeds to associate a person with a given device. You can also see which users accessed certain systems, applications and documents to determine which resource\u2014such as a PDF\u2014was the cause of the harm.\u201dA sophisticated attack might look suspicious but may not generate an alert. But if there is an egress to a known bad IP address, you could see that the IP address that the PDF came from matches, and then take the appropriate action.At the same time, it\u2019s important to not get caught up in alert fatigue, where high volumes of alerts could lead to an investigation that may take days or weeks to close, forcing you to miss the real attack taking place somewhere else. You need to be able to link the activity to some context so you can act in the best way possible at the best time possible.\u201cThe perfect storm isn't always the case, and your network operations team doesn't work 24\/7,\u201d Humphreys warns. \u201cYou should thus support the latest, most prominent firewalls and send a program command to block malicious traffic temporarily. You have to use the tools in context and in a smart, automated way.\u201dThe value of operationalizing threat intelligence\u2014in contextMany large companies position themselves as keepers of global intelligence because they have thousands of customers with tens of thousands of nodes, and they share the data with other companies. Ingesting and digesting this data and then relying on only signature-based and rules-based solutions means that constantly morphing malware can easily squeak by. \u201cThis isn\u2019t operationalizing\u201d, argues Clementelli.As you plan to operationalize your threat intelligence program, keep in mind that the value of threat intel is only as good as the sources of the data and the programs the data feeds. A good analytics engine fed with bad data is not as good as a decent analytics engine with credible, relevant intelligence. Context has to be built around the other variables you can also see\u2014security analysis requires more than pure security data.When taking on this challenge, you will most likely need to identify and collaborate with a security expert trained in big data and security analytics. Similarly, be sure to identify solution providers and security vendors that can provide expertise in both internal and third-party vendor risk management as well as security incident response. It\u2019s critical to thwart as many attacks as you possibly can up and down the supply chain, but when attacks succeed, it\u2019s just as important to limit the damage and immediately return your network infrastructure to normal operations and to a fully secure state.Sean Martin is a four-term CISSP and 25-year information technology and information security veteran. Write to him at firstname.lastname@example.org.