Backups aren’t just for smoking crater scenarios

Every company I had worked for in the past was another piece in my continuing education. Along the way there have been some lessons that were recurring. One of the main ones was around backups. Time and again I would encounter the most curious backup…um, strategies.

At one company in particular I made the mistake of asking what we were doing for backups on the core production systems. I was met with confused looks and the response that any server could be rebuilt by reinstalling the operating system. I asked about the database and was met with a glazed over look.

This was a shop that had absolutely no backup plan whatsoever. When I dug a little deeper I discovered that only some systems were being backed up at all. And none of those backup tapes were ever tested. No one knew if the tapes would even recover a single iota of data. But, the rationale was that the systems were being backed up and thus, compliant.

This was little more than malicious compliance. Why would they bother to backup the systems simply to satisfy a tick box on a compliance check list? Seemed like a wasted effort. Why would a company not have an a sound backup strategy? The thought process that it would be simpler to just do the bare minimum has always confused me.

Only a few core systems were being backed up. None of the tapes had ever been tested and no one seemed to care. It was a mystery. Have you ever tested your backups? This is a question that keeps surfacing and, much like patching regimens, people seem to pay it little more than lip service.

At another organization I asked a different question. I asked how we secured backups. Were they stored offsite? Were they encrypted? Again, more blank stares. I was told that we didn’t need to encrypt the backups as the tape system that we used to do backups was so old no one would have it.

Within 2 minutes I found the exact tape library system on eBay for little more than the cost of an extra large pizza and a six pack. The ostrich approach to dealing with backup related issues had failed yet again. Yes, I do realize that ostriches do not actually bury their heads in the sand.

The one common theme after backup plans and storage / encryption that was always present was the failure to test backups. In all the instances where I ran into this sort of curious behavior I never once found a case where the backups were ever tested. So, heaven forbid, the favorite disaster recovery scenario of the “smoking crater” were to occur. If that came to fruition there would be a better than zero chance that the recovery time objectives (RTO) would not be met.

Backups are there for your benefit as well as that of your users / customers. Doing the bare minimum to pass an audit does you no favours. Not having a clear plan will most likely end in ruin. Not having a sound storage and encryption regimen is something that one can ill afford. If you are taking backups, be sure to test them regularly to ensure they would actually work if and when you might need them.

A novel idea, I know.

For those of you who have transitioned to a cloud environment. I’ll talk to you about locking down your systems another time. For now, I have to go spend some money on eBay.