Incident with Codespaces

Resolved

This incident has been resolved.

Investigating

On 2024-05-28 between 18:30 and 21:30 UTC, the Codespaces service was degraded. Users with dev container configurations that depended on the DockerHub container registry were unable to create new codespaces unless they provided their own DockerHub registry credentials. We observed a rise of at most 6% of creations entering recovery mode due to this incident.The GitHub Codespaces team worked directly with Docker to mitigate and has engaged with Docker to prevent similar issues in the future.

Investigating

A fix has been applied and we are seeing some recovery. We will continue to monitor for a bit before marking this issue resolved.

Investigating

We are still investigating root cause and remediation options. In the meantime, here is a workaround to be able to pull images from DockerHub:1. Make a free DockerHub account at https://hub.docker.com (or use an existing account if you have one).2. Create a DockerHub secret/PAT from https://hub.docker.com/settings/security (Read permission should be sufficient).3. Go to https://github.com/settings/codespacesAdd three Codespace secrets:- DOCKERHUB_CONTAINER_REGISTRY_PASSWORD (equal to the DockerHub PAT you created)- DOCKERHUB_CONTAINER_REGISTRY_SERVER (equal to https://index.docker.io/v1/)- DOCKERHUB_CONTAINER_REGISTRY_USER (equal to your DockerHub username)4. Make sure these secrets are set as visible to the target repo.5. Create/rebuild your CodespaceSteps above are distilled from the official docs: https://docs.github.com/en/codespaces/reference/allowing-your-codespace-to-access-a-private-registry#example-secrets

Investigating

Duplicate update, same as above

Investigating

Some Codespaces are currently failing to be properly created for images hosted by DockerHub. Other registries should be unaffected. We are investigating root cause and will report back shortly.

Investigating

We are investigating reports of degraded performance for Codespaces