A lot of people talk about disaster recovery and not many people get how it fits into the business and how or why it can be important, we thought we would share a recent chain of events that worked so well most of the users in the business didn’t know anything was going on. Sometimes if you do your job well enough no one knows your doing it!
One of our clients is a Tier 1 automotive manufacture supplying to internationally known Car manufactures. All companies operate 24/7 so there is a strict zero downtime policy. Every minute of downtime can equal hundreds of thousands of pounds of losses and extra costs for staff to catch up on hours lost in the factories.
We had been planning a virtualisation and disaster recovery project for all the physical servers, so when we discovered we were also moving to a new large factory and office we tied the projects together. We virtualised all physical servers and even some routers. Once virtualised we enabled off site replication to a secure Datacentre, this allowed us to turn off physical hardware and move the office without any of the key ERP software and backbone software like Emails being interrupted, despite the original hardware being in the back of a lorry being moved!
After the original physical servers had arrived at the new location we switched back to the main hardware’s new location. Now the disaster recovery plan had been tested in a controlled manor we felt comfy enough after additional testing that in event of a disaster at the main site the business would be able to ride through large problems which could have crippled the company before.
The disaster recovery plan was tested for real shortly after being at the new factory when the sites power lines failed due to circumstances outside of the clients control, at which point generators kept the factory running, while the company disaster recovery plan kept all key ERP software running and externally all emails and VoIP phone lines stayed up. Externally nobody knew anything and more importantly production was able to keep running.
If the downtime had been for a longer period of time the plan would have allowed all employees to go home and log onto the sessions via remote desktop and carry on working where they left off. Phone extensions could have been diverted to user’s mobiles but luckily for all involved the downtime was not prolonged.
The one thing which a good plan always seems to lack is staff training. Having a plan is great but people need to know it, when it kicks in and what they need to do, so drumming in basics like accessing emails and phones is key.
With ERP software becoming so important to large businesses, having multiple plans to survive basic incidents is a must. SaaS is something we expect to be seeing more of in the future and while some people are sceptical of SaaS, cloud and disaster recovery being buzzwords, they can boil down to some very simple procedures and just using what you may already be using. Businesses are starting to see that disaster recovery systems combined with virtualisation and cloud is a very smart and powerful tool which can be applied to all large and small businesses.