Intermittent inbound/outbound call errors
Incident Report for SpecTronics UK
Postmortem

Yesterday morning we were responding to a server specific issue that required unplanned maintenance at our London datacentre which coincided with an unforeseen increase in compute load on our infrastructure. The combination of removing a portion of our capacity for maintenance coupled with an upswing in compute demand led to a requirement of a full restart of a number of impacted servers. While these systems were being restarted, our platform was unable to provide service which caused the resultant, brief outage of 17 minutes from 10.08.

Our platform is architected to support the removal or failure of significant capacity with little or no customer impact. We build our systems with the assumption that things will occasionally fail, and we rely on the ability to remove and replace capacity as one of our core operational processes. While this is an operation that we have relied on to maintain our systems since the launch of our platform, it has on this rare occasion not delivered a continuation of service.

We have already modified a number of procedures and have added additional safeguards to further reduce the chance of this situation occurring in the future.

While we are proud of our long track record of reliability and availability, we know any disruption to this service is frustrating for you. We feel it too because we use the service every day, so we know how disappointing it can be.

We would like to take this opportunity to thank you for your continued support as we move forward into 2018 and wish you all an enjoyable festive period.

Posted over 1 year ago. Dec 14, 2017 - 19:24 GMT

Resolved
We are confident that the problems which affected the customers VoiP services between 10:20am and 10:40am today have now been resolved.

All servers are now fully back online and handling a normal volume of calls.

This service report will now be closed. Again, we apologise for any inconveniences caused.
Posted over 1 year ago. Dec 13, 2017 - 16:43 GMT
Monitoring
Following our earlier update all servers are now fully back online and handling a normal volume of calls.

We will continue to monitor the platform for observational purposes before closing the status.

Again, we apologise for any inconveniences caused.
Posted over 1 year ago. Dec 13, 2017 - 11:35 GMT
Investigating
We are currently investigating an intermittent problem within inbound and outbound calls on the VoIP platform.

Our engineers have identified the cause of the problem and will be working on restoring the service to its original state as a critical priority.

We will update this status as soon as updates become available.

We apologise for any inconveniences caused.
Posted over 1 year ago. Dec 13, 2017 - 10:52 GMT