Resolved -
All backlog processing has been completed and services remain stable. This issue has now been resolved.
Our teams will continue to monitor the platform to ensure ongoing stability. Further details will be shared in a post mortem report.
May 14, 21:42 PDT
Update -
Most data processing has completed, and the platform continues to operate in a stable state while keeping up with incoming data. Our teams are performing final validations to confirm everything is functioning as expected.
We will provide a final update once these validations are complete.
May 14, 18:30 PDT
Update -
Processing continues following an overnight service restart that required a portion of data to be replayed. Most organizations are now processing within approximately one hour of real time, and the remaining backlog continues to reduce steadily.
Our teams are actively monitoring the environment to ensure full recovery and continued platform stability.
May 13, 23:54 PDT
Update -
Most data is now being processed successfully, and the platform continues to operate in a stable state while keeping up with incoming data. A small amount of remaining backlog is still being worked through, with recovery activities managed carefully to maintain overall service stability.
We will continue to monitor progress and provide additional updates as recovery completes.
May 13, 09:07 PDT
Update -
Our technical teams continue to closely monitor recovery progress. The system is steadily processing the existing backlog in a controlled manner, with priority placed on maintaining platform stability.
We will continue to provide updates as recovery progresses or if there are any material changes.
May 13, 03:21 PDT
Update -
Backlog processing continues, and the platform remains stable. The system is keeping up with incoming data while previously delayed inventory data is being processed in a controlled manner to ensure overall service stability.
We will continue to monitor progress and provide additional updates as recovery continues.
May 12, 17:45 PDT
Update -
Backlog processing remains in progress, and the platform continues to operate in a stable state. Recovery activities are being carefully managed to ensure delayed inventory data is processed safely while maintaining overall service stability.
We will continue to monitor progress and provide additional updates as recovery continues.
May 12, 07:39 PDT
Update -
Backlog processing continues to make steady progress following the previously deployed fix. The platform remains stable, and recovery activities are being carefully managed to ensure delayed inventory data is processed safely without impacting downstream services.
Customers may continue to experience delays in updated inventory data appearing during this recovery period. We will continue to monitor progress and provide further updates as appropriate.
May 11, 09:58 PDT
Update -
Our technical teams continue to work through the backlog recovery process. Recovery activities are being carried out cautiously to avoid overwhelming downstream services and to ensure platform stability throughout the restoration process.
May 11, 04:16 PDT
Monitoring -
Our technical teams have successfully designed and implemented a fix. Services are now operating normally, and backlog processing is underway for the data accumulated during the incident.
May 10, 22:06 PDT
Investigating -
Incident Description: We are investigating an issue impacting IT Visibility (ITV) services in the US Production environment. Affected customers may experience delays in normalized inventory updates appearing in ITV .
Priority: P2
Restoration Activity: Our technical teams have identified that a backend service is experiencing elevated request throttling from an upstream cloud provider dependency, resulting in delayed inventory processing. Teams are actively investigating the issue and working to restore normal processing. We are monitoring the environment closely and will continue to provide updates as progress is made toward full restoration.
May 10, 20:31 PDT