Inaccurate call stats in Agent monitor
Incident Report for Sangoma
Postmortem

Earlier today, there was a hiccup with one of our backend systems, specifically a cron worker. This issue caused a temporary delay in updating the Agents Monitor. We want to assure you that during this time, there was no impact on your calls or the quality of service provided.

Our vigilant team quickly identified the root cause — a dead cron worker that slipped past our monitoring system — and took immediate action to rectify the situation. By restarting the service and deploying additional workers to manage the backlog of events, we were able to normalize operations within just 30 minutes.

Rest assured, our top priority is always to ensure the smooth functioning of our systems and to minimize any disruption to your experience. We've implemented measures to prevent similar incidents in the future, including enhancing our monitoring capabilities to better detect issues like this.

If you have any questions or concerns, please don't hesitate to reach out to our support team. We sincerely apologize for any inconvenience this may have caused and appreciate your understanding as we continue to improve our services.

Posted Apr 04, 2024 - 15:30 EDT

Resolved
Earlier today, there was a hiccup with one of our backend systems, specifically a cron worker. This issue caused a temporary delay in updating the Agents Monitor. We want to assure you that during this time, there was no impact on your calls or the quality of service provided.

Our vigilant team quickly identified the root cause — a dead cron worker that slipped past our monitoring system — and took immediate action to rectify the situation. By restarting the service and deploying additional workers to manage the backlog of events, we were able to normalize operations within just 30 minutes.

Rest assured, our top priority is always to ensure the smooth functioning of our systems and to minimize any disruption to your experience. We've implemented measures to prevent similar incidents in the future, including enhancing our monitoring capabilities to better detect issues like this.

If you have any questions or concerns, please don't hesitate to reach out to our support team. We sincerely apologize for any inconvenience this may have caused and appreciate your understanding as we continue to improve our services.
Posted Apr 04, 2024 - 15:30 EDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Apr 04, 2024 - 11:10 EDT
Investigating
We are currently investigating an issue concerning inaccurate statistics displaying on the agents monitor in CX.
Posted Apr 04, 2024 - 09:45 EDT
This incident affected: Business Voice Unified Communications Platform (UCaaS) (Sangoma CX (formerly known as Sangoma Contact Center / SC3)).