Outbound message delivery is delayed/slow for some SparkPost and SparkPost Enterprise customers (US hosted only)

Incident Report for SparkPost

Postmortem

We have completed our analysis of last Thursday’s incident that impacted some SparkPost customers provisioned in the US region.

During the incident period (20:10 UTC - 21:40 UTC), some customers experienced latency and observed an increase in 5xx error rates on calls to the Transmissions API. Additionally, some customers observed delays - up to 75 minutes in some cases - on outbound message delivery. The incident impacted only a portion of our platform tier, so many customers would not have experienced latency, errors or delay of outbound message delivery.

This incident was precipitated by an unexpected customer usage pattern that was not fully accommodated by the API service tier. We have put safeguards in place to protect against this specific scenario and we will perform a review of the entire API service tier by the end of this week to identify any usage patterns that present similar risk. We are sorry for any inconvenience this caused.

Posted Jul 20, 2020 - 09:05 EDT

Resolved

This incident has been resolved.
Posted Jul 16, 2020 - 19:02 EDT

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Jul 16, 2020 - 17:57 EDT

Update

We are continuing to monitor the situation
Posted Jul 16, 2020 - 17:46 EDT

Investigating

We are investigating an increase in delivery latency for some outbound messages. (NOTE: This does not impact our customers hosted in the EU.)
Posted Jul 16, 2020 - 17:35 EDT
This incident affected: SMTP Delivery (Outbound Message Delivery) (SMTP Delivery - USA).