[L66#2528027] Physical Interfac to RETN is Down!

Incident Report for level66.network

Postmortem

We recieved the following RCA by RETN:

Service interruption was caused by a software bug on our switch in Frankfurt, which caused the unreachability of the equipment. In order to restore the service, our engineer replaced the faulty equipment. No further interruptions are expected.

Posted Dec 01, 2023 - 08:56 UTC

Resolved

Update by RETN: All services have been restored.

Posted Nov 26, 2023 - 21:06 UTC

Update

Update by RETN: We observe that not all services were restored after the works. The reasons are being clarified.

Posted Nov 26, 2023 - 20:48 UTC

Monitoring

Update by RETN: Restoration work has been completed. No further interruptions are expected.

Posted Nov 26, 2023 - 20:17 UTC

Update

Update by RETN: Our engineer has arrived to the site. The ETR is 22:00 UTC+2.

Posted Nov 26, 2023 - 18:31 UTC

Update

Update by RETN: Our engineer is still en route and approaching the incident site. Arrival time remains the same, we will notify you when we have new information.

Posted Nov 26, 2023 - 17:55 UTC

Update

Update by RETN: We are experiencing unavailability of our equipment in Frankfurt. Our engineer is currently travelling to the incident site. The expected time of arrival of the engineer is 20:00 UTC +2. We will keep you informed as information becomes available.

Posted Nov 26, 2023 - 16:11 UTC

Identified

Update by RETN: We observe the unavailability of our equipment in Frankfurt.
Start Time: 26.11.2023 15:26 GMT
Location: Germany, Frankfurt am Main
We apologize for any possible inconvenience.

Posted Nov 26, 2023 - 15:54 UTC

Investigating

We do not see any light from RETN anymore. We've already opened a incident ticket on their end and are currently awaiting their feedback.

Posted Nov 26, 2023 - 15:34 UTC

This incident affected: Transit (AS9002 / RETN).