On September 7, 8 and 13 we reported an increase in delivery latency for outbound messages for customers in the US region. This impacted some but not all customers. Impacted customers observed a longer-than-usual time between injection and delivery for some messages during the incident periods.
The root cause of these incidents was a network capacity limitation. During the incidents we implemented a number of short term mitigation steps to optimize delivery rates (including rebalancing customer traffic) in order to reduce the impact to customers. On the evening of September 14 we completed the long term solution: a physical upgrade of our edge-of-network tier. We are confident the current architecture will support projected growth models into 2025.
Posted Sep 20, 2023 - 08:52 EDT
Resolved
This incident has been resolved.
Posted Sep 13, 2023 - 13:33 EDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Sep 13, 2023 - 12:51 EDT
Identified
The issue has been identified and a fix is being implemented.
Posted Sep 13, 2023 - 12:16 EDT
Investigating
We are investigating an increase in delivery latency for some outbound messages. (NOTE: This does not impact our customers hosted in the EU.)