Explosion at HSNTX1 - Aftermath
Well, we finally have everything back online after the explosion and subsequent fire at HSNTX1. Typically enough, we had just finished migrating 5background-color: #b7c3cf; -moz-border-radius: 10px; min-height: 20px; width: 575px; font-weight: normal; padding: 15px 15px 15px 15px;+ websites to new servers, when the datacentre finally came back online. So if we had done nothing at all, we would be in a similar position to where we are now. But who was to know that?
Explosions in datacentres are pretty rare occurrences. But it is something we will have to bear in mind, having experienced it once. At least our backups were good, and it provided a good test of our systems.
As for The Planet, one of the largest datacenter companies in the US, the disruption for them will last much longer, and the repercussions will be much more serious. The loss of business and damage to reputation will also be tremendous.
Sadly, we were aware of the catastrophe several hours before the Planet kicked into action. Our SMS monitoring system let us know that two servers were down immediately, and logging in to the forums just before midnight we could see that around 1background-color: #b7c3cf; -moz-border-radius: 10px; min-height: 20px; width: 575px; font-weight: normal; padding: 15px 15px 15px 15px;background-color: #b7c3cf; -moz-border-radius: 10px; min-height: 20px; width: 575px; font-weight: normal; padding: 15px 15px 15px 15px; others were also aware. However, the first statement from thePlanet came several hours later, and it seemed to take them several hours more before a full mobilization occurred. By Sunday morning, almost 4,background-color: #b7c3cf; -moz-border-radius: 10px; min-height: 20px; width: 575px; font-weight: normal; padding: 15px 15px 15px 15px;background-color: #b7c3cf; -moz-border-radius: 10px; min-height: 20px; width: 575px; font-weight: normal; padding: 15px 15px 15px 15px;background-color: #b7c3cf; -moz-border-radius: 10px; min-height: 20px; width: 575px; font-weight: normal; padding: 15px 15px 15px 15px; people were logged in clamouring for news.
Personally I won't complain about that delay, since midnight on Saturdays most people have better things to do than to worry about exploding power plants.
Anyway, just glad the drama is over for us at least, till the next time ....
