Data processing delays for SparkPost customers - US region

Incident Report for SparkPost

Resolved

This incident has been resolved.
Posted Mar 07, 2023 - 17:39 EST

Monitoring

The backlog of message event data for customer webhooks and the Events API is caught up for most customers. However, the metrics data for the Signals reports is still backed up; we expect it to be fully caught up for all customers in approximately 2 hours.
Posted Mar 07, 2023 - 16:17 EST

Update

We have rolled out a fix and are making good progress on getting through the backlog of data. We expect the queues to be cleared for all customers in approximately 2 hours. Next update ~21:00 UTC
Posted Mar 07, 2023 - 14:55 EST

Update

We have rolled out a fix and we are starting to process the backlog of data.
Posted Mar 07, 2023 - 13:44 EST

Update

We are continuing to work on a fix for this issue. Next update will be no later than 1 hour from now 19:30 UTC.
Posted Mar 07, 2023 - 13:25 EST

Update

We are continuing to work on a fix for this issue. Next update will be no later than 1 hour from now 18:30 UTC.
Posted Mar 07, 2023 - 12:33 EST

Identified

The issue has been identified and we are working on a fix.
Posted Mar 07, 2023 - 11:15 EST

Investigating

Our data processing services are running behind; some customers may see a delay of data available via the:

SparkPost App (webUI): Signals Reports
Events API
Customer webhooks endpoints

Message injection and outbound message delivery are not impacted - all messages are flowing as expected.
Note: This issue does not impact our SparkPost customers hosted in the EU.
Posted Mar 07, 2023 - 11:09 EST
This incident affected: Metrics API (Metrics API - USA), Events API (Events API - USA), SparkPost Application (WebUI) (SparkPost Application - USA), and Event Webhook Delivery Service (Event Webhooks - USA).