Cloud-based Security Event Management?

Security Incident and Event Management (SIEM, though some people still use SIM or SEM) has been around for about a decade now. Originally built to capture, normalize, process, and correlate firewall and IDS logs, it is now participates more extensively in areas like log management, security analysis and forensics, regulatory compliance, and risk management. As SIEM evolved, it has been called upon to collect and analyze more data from more devices across the enterprise. At the same time, vendors added additional functionality for storage management, data analysis, and reporting. Taken together, these new requirements and product enhancements have created a major bottleneck where many SIEMs can no longer keep up with today’s needs for massive scalability. This issues is felt most acutely with event correlation. Most SIEM platforms simply can’t process and analyze the massive number of log events generated in real-time. Looking ahead, this problem will likely get worse. Why? Server virtualization is multiplying the number of assets generating events in the data center. Users are bringing in new device types. Security tools will also go virtual thus increasing the number of events generated. Finally, there are new types IT data for analysis like the Trusted Computing Group’s (TCG) Interface for Metadata Access Points (IF-MAP), and the recently discussed AppFlow being pushed by Citrix and others.All of this means that SIEM is quickly approaching supercomputer-like requirements. Okay, so what can be done about this. Well there are a few vendors like NitroSecurity and SenSage that anticipated this type of requirement and built highly scalable products. Others are responding to this problem and taking traditional scalable transactional application approaches. Like what? They are modifying their application architecture to take advantage of multi-threading on new multi-core processors. They are moving to 64-bit architectures to get beyond 32-bit memory limitations. They are parallelizing their applications to run on server clusters. These are intelligent steps but they are exactly the same things you would have done 20 years ago. In fact, Oracle Cluster Server (OCS) took these steps back in the early 1990s and ran most effectively on Sequent servers (Yes, I’m showing my age here). I have an alternative suggestion for the security management industry; it’s called “cloud computing.” Why not take advantage of the massive processing scale of say AWS and build a burstable hybrid-cloud application architecture for SIEM? The model certainly works, just ask Facebook games maker Zynga about architecting a distributed application architecture that leverages AWS for massive scale.Yes, I know this is security data but remember that log events aren’t the same as confidential files or private data from a database column. SIEM is a resource-intensive application that will continuously require more and more processors and storage. AWS offers these resources at a fraction of what they would cost an enterprise to purchase, deploy, and manage. Given this, cloud-based SIEM processing seems like a match made in heaven to me.