Be afraid of the dark web – or learn to monitor it

In recent months, many column inches have been dedicated to discussion of both the deep web and the dark web. It’s very important to draw a distinction between the two. The deep web usually refers to sites that aren’t indexed by search engines, while the dark web is primarily comprised of sites that need software such as Tor or the Invisible Internet Project (I2P) to access. In this article, we’ll focus on the dark web, which news articles often paint as the seedy underbelly of the internet – a place where no respectable browser will dare venture.

While it is true that there are many sites on the dark web dedicated to the buying and selling of illegal items, such as drugs, weapons and even human organs, it is worth bearing in mind that there are also some positive, legal reasons that people use the dark web. For example, activists working under oppressive regimes will often use the dark web to communicate safely and anonymously. The dark web also serves as an ideal platform to facilitate and protect the freedom of speech of individuals, particularly those facing oppression or persecution for expressing their opinions.

However, though there are some positive use cases, the dark web is frequently used for nefarious purposes. For example, if information is stolen in a data breach, it is almost guaranteed to end up for sale on the dark web. The exchange of such information – which can include customers’ personal information and credit card details, or even confidential corporate data – is rapidly becoming a huge business opportunity for cybercriminals.

Therefore, it’s critical that organizations and security researchers obtain insight into what goes on within the dark web, and get alerted right away if there is any information being exchanged or discussed that relates directly to them. Speed is usually of the essence here, as compromised credentials can quickly change hands and be used to take over accounts. However, achieving the level of visibility into dark web transactions required for such detection is far easier said than done.

Challenges of gaining visibility into the dark web

Like the deep web, the dark web cannot be crawled and indexed by search engines, so finding relevant data isn’t as simple as running a search engine query. Dark web users must manually identify the dark web nodes that have relevant information.

After the nodes of interest have been identified, the next step is to gain access to them. This too can be a hurdle, as sites are rarely open and generally require users to log in, which is not as simple as it is on mainstream websites. To gain membership, you must pass through a thorough vetting process, and, often, it is also the case that an existing and trusted member of the site must recommend you.

Last, but certainly not least, language barriers can be another major challenge. Dark node operators speak a variety of different languages, making communication difficult if they don’t speak your language.

Even if you successfully jump through these hoops, dark node operators can retract your access at their own discretion and at any time. For example, they may block users they suspect of monitoring their activities as part of a law enforcement operation.

Collecting dark web data

Once you can gain access to the dark web node, then the next step is acquiring its data. The method for doing so resembles that of traditional threat intelligence gathering processes (as it combines both human elements and technology). Because most attacks against enterprises typically involve account or identity takeover, the most commonly sought after (and useful) forms of information are user credentials or personally identifiable information (PII). Here’s a look at the typical steps involved in monitoring for and collecting this data from the dark web:

• Parsing: There are usually large amounts of data that need to be initially parsed. Technology is very helpful to automate this step, coupled with human validation.
• Normalization: After parsing, the data should be normalized so that it can be sorted and queried easily later in the process. This is also a good time to deduplicate as well as remove all records that do not contain relevant data.
• Validation: After data has been normalized and deduplicated, it is smart to run validation against the data to ensure it is accurate.
• Refinement and enrichment: At this point, the data is ready to be used – although many companies will opt for further refinement and enrichment to add contextual information that makes the data relevant to their organization and risk profile.

How organizations can protect themselves

While the dark web in and of itself may not be a threat to organizations, the ever-increasing amount of stolen corporate data offered within it means that it’s becoming more and more critical that companies find ways to monitor it.

How closely the dark web needs to be monitored depends on each organization’s own capabilities and appetite for risk. However, all organizations must follow the same basic security fundamentals and best practices: namely, to achieve an understanding of who their typical adversaries are, what their motivations are and which types of data they are likely to target.

As monitoring and collecting dark web data can be labor-intensive, it often makes sense to outsource the activity to a specialist company that can monitor the dark web on your behalf, and provide alerts if any employee or customer data is being actively traded.

However, as with all threat monitoring, the solution isn’t merely obtaining situational awareness about the threat landscape. Companies also need to have adequate incident response and recovery controls and procedures in place to respond appropriately and in a timely fashion if credentials are stolen so that they can minimize the impact of any attack.