Incident with Webhooks

Resolved

On January 14, 2026, between 19:34 UTC and 21:36 UTC, the Webhooks service experienced a degradation that delayed delivery of some webhooks. During this window, a subset of webhook deliveries that encountered proxy tunnel errors on their initial delivery attempt were delayed by more than two minutes. The root cause was a recent code change that added additional retry attempts for this specific error condition, which increased delivery times for affected webhooks. Previously, webhook deliveries encountering this error would not have been delivered.The incident was mitigated by rolling back the change, restoring normal webhook delivery. As a corrective action, we will update our monitoring to measure the webhook delivery latency critical path, ensuring that incidents are accurately scoped to this workflow.

Investigating

Some webhook deliveries are delayed, but we don’t expect meaningful user impact. The delays are currently scoped only to deliveries that, until recently, would have failed more quickly. We will update status if conditions change.

Investigating

We are investigating reports of degraded performance for Webhooks