When big data gets personal…and cloudy

According to IBM, we create 2.5 quintillion bytes of data per day. That is a lot of really big data. And big data is getting bigger. It is becoming even more ubiquitous as our Personally Identifiable Information (PII) not only forms part of it, but also helps to create ties between other information, like marketing data. Big data is spreading, like a slime mold across our web servers, into our mobile apps, and IoT devices; it is everywhere and it represents us, the individual.

All of these data have to be stored somewhere and made accessible somehow. The records containing not only our PII, but health data, marketing preferences, and much more, end up in Cloud repositories. Stored using the likes of Amazon storage services, such as S3, Elastic File System (EFS) and Elastic Block Store (EBS) or Microsoft Azure Blob or Google Cloud Storage. According to research by Global Market Insights, the Cloud IAM market is said to be worth almost $27 billion by 2022 with Cloud storage being increasingly used across the company size divide – the industry seeing 40%+ year on year growth according to analysts Synergy. That’s an awful lot of personal data stored in the Cloud.

Trouble is, the data is very attractive to the average cybercriminal. There was a 40% increase in identity thefts in 2016 according to ITRC, and of the 1.38 billion data records stolen in 2016, only 4% were encrypted. But, cloud-based information needs to be legitimately accessed too. Consumer IAM systems often have millions of users, and the load on the system has to be finely, yet robustly, balanced.

With PII breaches like Equifax still fresh in our minds, we need to look seriously at how we build Cloud-based identity services (IDaaS) for mass adoption, making sure the PII of each record is protected when stored, yet retaining high availability.

IDaaS storage – the balance between elastic scalability and security

If you have existing accounts and want to migrate them to Cloud storage when you move over to an IDaaS solution, you aren’t alone. According to Amazon, so far, 40,000 have migrated across to AWS. Moving from on-premise to Cloud storage for your IAM system will take knowledge and resources to get it right. Companies like NetApp have sound advice on how to do it smoothly, but if you want to extend your IAM to a Consumer IAM (CIAM) you will have to make that move; the Cloud gives you the elasticity and flexibility to service a customer-facing IAM platform that may have many millions of users with spikes and troughs in usage.

Whether you are starting from scratch or migrating from on-premise to Cloud, choosing the right Cloud storage is vital for identity-based services. Consumer-facing identity systems are the very definition of ‘elastic’. Take a service which allows you to lodge a tax claim. There are certain times of year where the identity system behind the tax service will be heavily used, often by many millions of users. Or student services, where September may be a very busy month, but February less so. If the Cloud storage solution you use is not able to adapt to these changeable volumes, it will either crash or cost you too much to run.

Elasticity is vital, but so is the security of PII. And, as with everything in the world of security, there is a balance to be met. If we want to ensure that your customer PII is protected you have to get the Cloud security aspect correct too.

As an example, if we take the Amazon Cloud storage options, S3, EFS, and EBS, at first glance they may seem similar. All three offer AES 256 bit encryption for data at rest. All three have decent scalability (some more elastic than others). But look under the cover. There are subtle differences that need to be explicitly reviewed to find the right fit for your purpose. Amazon EFS fits the remit of elastic scalability for mass adopted CIAM because it is able to handle massive amounts of data across multiple instances. S3 however, fully supports compliance for HIPAA and PCI-DSS. Finding the right balance for your particular setup is a challenge, but an important one to meet.

But remember, security isn’t just about protecting data from malicious exposure. It is also about protecting against data corruption and loss. A CIAM system has to be highly robust and choosing the right Cloud infrastructure for your particular service is vital in maintaining integrity.

Security considerations

Security is of course, not just about protecting a single point of failure. It cuts across all aspects of a CIAM system. But in terms of choosing the right Cloud storage, security check boxes need to be met:

Data at rest protection

Modern Cloud storage solutions will offer AES 256 encryption as a default. However, as always, the devil is in the detail.

Data in transit

TLS between the DB servers and application servers for data in transit is also essential.

Authentication and authorization

Your Cloud storage infrastructure is the heart of your operation – don’t let just anyone have access.

Backup and disaster recovery

You can imagine the scenario – you are running a business critical customer IAM service and suddenly vast numbers of customers are locked out of your service. Make sure the Cloud storage system has a seamless, well tested, process for this.

OWASP is also a good place to check out where the likely points of weakness in web applications are and in 2016, known vulnerabilities and misconfigured security were the top two data breach causes.

I’ll be looking at a secure approach to CIAM in more depth in a future post.

Enterprises are moving in their droves to a Cloud-centric IAM model. Consumer-facing IAM services and API-based identity services, are, without doubt, a driving force for full Cloud-Based IDaaS; keeping costs down, whilst optimizing security and elasticity is your challenge.