Incident with Copilot


Incident resolved in 2h19m7s

Resolved

On July 11, 2024, between 10:20 UTC and 14:00 UTC Copilot Chat was degraded and experienced intermittent timeouts. This only impacted requests routed to one of our service region providers. The error rate peaked at 10% for all requests and 9% of users. This was due to host upgrades in an upstream service provider. While this was a planned event, processes and tooling was not in place to anticipate and mitigate this downtime. We are working to improve our processes and tooling for future planned events and escalation paths with our upstream providers.

1720711277

Investigating

Copilot is operating normally.

1720711276

Investigating

We have mitigated the intermittent timeout errors impacting Copilot’s Chat functionality and expect the incident to be resolved shortly.

1720711189

Investigating

We continue to investigate the cause of intermittent timeouts impacting Copilot’s Chat functionality. This is impacting a small fraction of customers. The timeout errors we are seeing has reduced back to healthy levels for the last 60 minutes but we are monitoring closely.

1720710282

Investigating

We continue to investigate the cause of intermittent timeouts impacting Copilot’s Chat functionality. This is impacting a small fraction of customers. We will provide further updates as we continue resolving the issue.

1720707263

Investigating

We continue to investigate the cause of intermittent timeouts impacting Copilot’s Chat functionality. This is impacting a small fraction of customers. We will provide further updates as we continue resolving the issue.

1720704727

Investigating

Copilot's Chat functionality is experiencing intermittent timeouts, we are investigating the issue.

1720702937

Investigating

We are investigating reports of degraded performance for Copilot

1720702930