Continuous monitoring and web security: Are you competitive with Black Hats?

Continuous monitoring and web security
Credit: TVZ Design CC BY 2.0

Nowadays security companies have to compete not only with other vendors, but also with sophisticated cyber gangs. If you will fail to detect and patch security vulnerabilities in a timely manner – Black Hats will do it for you, but for their own ends

Continuous monitoring is becoming a very popular term, both among security vendors and CISOs. In a constantly changing and hostile network environment where new zero-day exploits appear almost every couple of days, continuous monitoring of your organization’s infrastructure is vitally important. The main role of continuous monitoring is to keep your security team constantly aware of newly detected vulnerabilities, weaknesses, missing patches and configuration flaws that appear to be exploitable.

[ ALSO ON CSO: How to maintain security in continuous deployment environments ]

Various products, solutions and services exist today to assure the continuous monitoring process within both large and small organizations. However, when examining the efficiency of such solutions, we should initially try to understand how competitive those solutions are on the market: and not [only] against other vendors’ solutions, but with Black Hats. Yes, you heard right – with Black Hats, who are also in competition when it comes to the timely and reliable detection of vulnerabilities in external infrastructure.

Jan Schreuder, partner, cybersecurity leader from PwC Switzerland, summarized the risks for businesses: “Cyber attackers operate in real time and are not waiting for organizations to complete their next vulnerability assessment or patch release cycle. Security risks are continually evolving and challenge organizations to look for ways to proactively identify, evaluate, and manage cyber risks.”

Let’s take corporate web applications as an example of competition between White Hats and Black Hats in the continuous monitoring business. Websites have become a very efficient starting point for Advanced Persistent Threats against organizations. Moreover, corporate web applications themselves can bring a great financial income to cybercriminals. A majority of web security researchers in recent years estimated that over 80 percent of websites are vulnerable, hackers could obviously not miss such a great opportunity to make easy money.

At my current work place, I have the opportunity to manage and participate in our daily practice of web penetration testing and managed vulnerability scanning. We often face compromised web applications and websites. While examining logs in order to understand how and when a system was compromised, we regularly face “continuous monitoring” by cyber gangs.

At a first glance, it’s not obvious to distinguish between various spam bots and simple hack crawlers launched by newbies, and professional scanning infrastructure implemented by Black Hats that can easily compete with major scanning vendors from Gartner’s Magic Quadrant.

I would slightly disagree that professional Black Hats use the same tools as penetration testers – they may rely on some open-source or commercial solutions (including free trials provided by many vendors), but they have the competency and money to develop their own tools “in-house” that perfectly suit their business needs that are quite different from the needs of penetration testing companies.

Server infrastructure for such monitoring can be easily purchased from VPS and cloud providers who fight among each other for new customers. Money-hungry hosting companies will often prefer to close their eyes as long as possible before suspending someone’s account. And even after suspension, hackers can easily recover their data from a [cloud] backup. Investigation of such cases is often pointless: cybercriminals have fully functional and totally anonymous payment methods. Usually money passes via several different payment options and systems, converting mobile payments into crypto currencies, before transferring them to a pre-paid credit card obtained online with a stolen or fake ID. Simplified e-payment and money transfer systems, kindly developed by financial institutions for developing countries, is a real gift for hackers.

Let’s take a closer look at the scanning infrastructure at the disposal of cybercriminals. Like cybersecurity vendors, cyber gangs specialize in different niches. There are hacking teams in charge of continuous scanning and identification of potential victims: they crawl the Web in real-time searching for vulnerable websites with high Google Page Rank or Alexa Rating.

Hacking teams have efficient fingerprinting technologies to detect installed web applications, their modules, versions and patches. In our experience, for this purpose hackers usually take open-source software as a scanning platform base, improve its scanning and vulnerability detection algorithms, and adopt it for their needs. Often they have dozens or even hundreds of synchronized servers, forming a powerful scanning infrastructure capable of monitoring whatever they need.

An interesting feature is a regularly updated black list of domains/IPs of honeypots, security companies or law enforcement agencies – hackers tend to avoid probing them so not to expose themselves. Another interesting feature are priority lists – a list of high-priority targets (e.g. large e-commerce businesses or highly popular websites) that will be probed before others when a new zero-day is released, again - to outperform the competition.

Black Hats are very quick to detect and exploit new vulnerabilities in your system. According to the Drupal Security Team “Automated attacks began compromising Drupal 7 websites [that were not patched or updated] within hours of the announcement of the vulnerability”. Drupal clearly said that every [Drupal] website should be considered compromised unless patched within the first few hours of the vulnerability being announced. Many security vendors would envy such rapidity.

Once a vulnerable software version is detected, an alert is sent to other hacking teams specialized in exploitation (larger hacking teams can afford to perform the entire process “in-house”).

Usually exploitation is done automatically, but the exploits used are pretty advanced: hackers have different attack scenarios for different conditions and configurations (e.g. simple WAF bypass, exploitation within a chroot, insufficient permissions to create temporary files, non-standard admin panel location, older versions of MySQL, etc).

Once compromised, the victim’s application will be backdoored and sold for further exploitation on the black market. Usually, backdoors are very complicated to detect, unless you know every single file in your web root and constantly verify their checksums, but unfortunately, such systems are very rare. Moreover, complicated web applications can be backdoored via databases, allowing unprivileged users to execute arbitrary code that is silently stored in a tiny table of the DB.

Sophisticated hacking teams even patch the vulnerabilities after successful exploitation to prevent their competitors from exploiting them. Yes, there is as tough a competition among cyber gangs as there is among cyber security vendors. Mainly such patches are done via an auto-update mechanisms (if available) or through non-suspicious changes, so website owners will not question why the new vulnerability does not affect their installation. Therefore, if you see that a well-known vulnerability is not exploitable in your system – it may be a bad sign.

Yes, properly implemented continuous monitoring is not an easy task. Jan Schreuder (PwC) summarizes the challenge for businesses: “In our experience the successful implementation of a continuous monitoring program often represents a significant change to the way IT departments operate, and to be successful it requires significant commitment through leadership support, enforcement, and system owner responsibility and accountability.”

However, if you don’t want Black Hats to monitor and patch your systems for you – take the time to implement continuous monitoring correctly within your organization.

This article is published as part of the IDG Contributor Network. Want to Join?

To comment on this article and other CSO content, visit our Facebook page or our Twitter stream.
Insider: Hacking the elections: myths and realities
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.