All Simple - [LON1] Maintenance on B-side power systems – Maintenance details
ns2 experiencing major outage
[LON1] Maintenance on B-side power systems
Completed
Scheduled for November 30, 2024 at 9:00 AM – 4:11 PM
Affects
Authoritative DNS
Under maintenance from 9:00 AM to 4:11 PM
ns1
Under maintenance from 9:00 AM to 4:11 PM
DNS Resolvers
Under maintenance from 9:00 AM to 4:11 PM
resolver1
Under maintenance from 9:00 AM to 4:11 PM
resolver2
Under maintenance from 9:00 AM to 4:11 PM
Virtual Server Hosts
Under maintenance from 9:00 AM to 4:11 PM
Updates
Completed
November 30, 2024 at 4:11 PM
Completed
November 30, 2024 at 4:11 PM
We have been advised that the maintenance is now complete and has been successful.
Update
November 30, 2024 at 3:04 PM
In progress
November 30, 2024 at 3:04 PM
The work is still ongoing, we have had the following update from our DC Operator, we will provide further updates when available:
All works are now complete, however during a firmware upgrade to the various components within the B-side UPS, we have run into an issue which means we still have this UPS in bypass mode at present.
We are working to a resolution on this as we speak, however at present, B-side power - while restored in terms of delivery to client racks - is running without UPS protection at this time.
So while full power has been restored to all racks (following the works to the B-side today), full restoration of UPS protection on the B-side is still pending at present.
Update
November 30, 2024 at 12:42 PM
In progress
November 30, 2024 at 12:42 PM
We can see the loss of power to the B-Side PDU, everything is working as expected and we are continuing to monitor the situation.
In progress
November 30, 2024 at 9:00 AM
In progress
November 30, 2024 at 9:00 AM
Maintenance is now in progress
Update
November 30, 2024 at 9:00 AM
Planned
November 30, 2024 at 9:00 AM
Our data centre provider will be running maintenance on their B-Side power systems during this window.
During the works they will shutdown power from the B-Side.
Their A-Side power infrastructure will be unaffected.
We are using redundant power suppliers in core network equipment and servers, the window is at-risk only.
Planned
November 29, 2024 at 4:33 PM
Planned
November 29, 2024 at 4:33 PM
In preparation for this work, supporting hardware has now had the primary power shifted to our A-Side PDU.