Is your resilience ever that resilient? Can you ever be sure that your failover will fail over or will your n plus one leave you with nothing? The problem with your business continuity is that it is almost impossible to test. Of course it works on paper and the salesperson isn’t wrong in trying to sell you the dream. In the same way, you are not wrong for trying to improve the odds.
If one thing has an expected rate of failure, then two has a lower risk. If you need two of something to cope, then get three just in case. Lay them out in a cluster. Get them to talk to each other and when one goes pop the others will happily take the strain. Or at least that is what we were told.
The theory is good yet the practice leaves a little to be desired. You see, in the end there is only one test and that is reality. Business continuity is unfair. You are never remembered for all the times that things continue to work but rather for the time they failed. A single failure can blemish an otherwise exemplary record. Information technology is supposed to just work.
They will use the email that failed-over or the telephony that is replicated to tell you about all of the systems that are not working. The irony will be lost.
Your continuity is planned in the office when things are calm and your business is progressing nicely. You plan for the expected and the likely and if you can afford it, the unexpected and the unlikely. You build in contingency, resiliency and redundancy. But continuity is played out in reality when things are manic and your business isn’t playing ball. It is the things that you hadn’t planned for that are going to trip you up.
Coincidences happen, that’s why there is a word for it in the dictionary.
All the work that we have done with our business continuity has been repaid handsomely. Systems have continued uninterrupted when problems have arisen yet there will always be something lurking in the fabric, some unknown trap ready to be sprung and catch you out.
A big thank you to all the support and the team who got us back up and running again.