Increase in Workflow runs stuck as pending

Resolved

This incident has been resolved.

Update

The fix has been fully rolled out and we are monitoring the results.

Update

We are currently rolling a fix for the issue. We are monitoring the results.

Update

We are still working on a fix for this issue. While we fix the issue, users experiencing pending Workflow runs can trigger an instant rollback to a working deployment created before Jun 24 21:00 UTC to resume operation on new runs.

Update

The issue has been identified. We are continuing to work on a fix for this issue.

Update

The issue has been identified and a fix is being implemented.

Investigating

We are currently investigating an issue where some recently deployed Workflow projects are stuck as pending and not delivering queued messages as expected.