Updated UK broadband provider Hyperoptic learned the importance of testing backup systems this week after the service went dark for customers in London.
The service fell over on November 11, and despite users reporting that Hyperoptic's status page claimed there were no issues, there were indeed.
A spokesperson for the company told The Register that "a small percentage of our customer base in North London experienced an unexpected temporary loss of service," but things were up and running again "within less than 24 hours."
According to the HyperopticCS account on Reddit, the issue was a little more widespread. One post noted: "The outage is affecting a wider network of exchanges throughout London, postcodes W2-W9 are all impacted to a degree."
Another stated: "The Zayo link is down and we're currently working with Openreach to get it fixed. Their engineers just arrived. All teams are now on site working diligently to get the problem sorted."
Zayo claims to be one of the largest global internet backbone companies. Losing the link meant a very bad day for Hyperoptic and its customers.
But wait, surely there was a backup? There was, but it appears it did not work as planned. A spokesperson told El Reg: "While our network is designed with built-in redundancy, we are also reviewing a secondary link that did not activate as expected, which should have limited customer impact."
And the primary link? "The outage was caused by damage to a fibre cable at our local aggregation point. Hyperoptic's engineering teams, along with our partners, worked tirelessly throughout the night to carry out repairs and restore connectivity as quickly as possible."
According to a Reddit post, service was restored by 0300 on November 12.
Despite the nearly day-long outage, Hyperoptic said customers won't receive compensation, as "this incident does not qualify for automatic compensation under the terms of Ofcom's voluntary scheme (of which Hyperoptic is a member)."
The company apologized for the disruption and described such incidents as "very rare," though questions remain about the backup system and why it did not activate when needed. ®
After publication of this article, a Hyperoptic spokesperson told The Register that "Zayo was not involved in the backup link or the failure in question." We're happy to clarify this.
Source: The register