As many of you know, Friday morning at around 3am Eastern Time, a fire broke out in Seattle at Internap which provides connectivity to the EchoDitto servers. To give you some quick background, EchoDitto uses Internap for co-location services; the Internap facility in Seattle is at Fisher Plaza, the only mission-critical business community in the Northwest.

Due to this emergency, all of the power to the hosting facility was cut for safety -- including backup power. This means that all of our hosted websites went down for approximately 24-36 hours. However, by yesterday afternoon several of our sites implemented temporary splash pages explaining the situation to visitors and redirecting as needed. The problem was very far reaching and affected many internet-related enterprises, including enterprise operations like authorize.net and Microsoft’s travel service website, bing.com/travel.

To be clear, at no time was any data, servers, or hardware at risk, nor were they damaged or compromised in any way. The main issue at hand during this situation was getting power back to the facility. Given the high profile nature of this outage, it received the highest level of attention from recovery teams on site. The EchoDitto team monitored the situation in real time and worked aggressively to transfer all of our services to a backup facility.

Perhaps not surprisingly, Twitter played a large role in communication regarding this event. Hashtags like #fisherfire, #fisherplazafire, and the ever misleading #serverbbq (misleading, of course, because the servers themselves were never "bbq’d") helped everyone effected by this event communicate efficiently with each other from the ground in Seattle to various locations, including EchoDitto HQ.

We have currently restored all websites affected by this outage. We are also working to reinforce data center redundancy for all of our sites should a situation like this arise in the future.

If you would like more information on this situation please email info@echoditto.com

Happy July 4th and enjoy the rest of your weekend.