fix(ci): route to dedicated in-cluster runner, fix DOCKER_HOST for DinD #3

Merged

forgeadmin merged 1 commit from fix/ci-runner-dedicated-dind into main

2026-07-12 15:05:38 +00:00

forgeadmin commented

2026-07-12 14:58:19 +00:00

Contributor

Infra side (terraform repo): new dedicated Forgejo Actions runner in-cluster on pve92 (59GB free RAM), with a proper Docker-in-Docker sidecar replacing the fragile rootless-Podman-on-a-starved-VM setup.

This PR (app side):

runs-on: self-hosted -> xi2ix-website (the generic label let this job land on runners with no Docker capability)
DOCKER_HOST -> the DinD sidecar's plain Docker socket, with .toolchain/env.sh auto-detecting so local dev (rootless Podman) keeps working unchanged

Root cause of the outage tracked in .planning/debug/ci-cd-tool-provisioning.md, confirmed live: VM 603 (the sole runner) has 1.9GB RAM / 92MB free, shared with infra-terraform's own CI -- explains every failure signature in that investigation. Infra side (terraform repo): new dedicated Forgejo Actions runner in-cluster on pve92 (59GB free RAM), with a proper Docker-in-Docker sidecar replacing the fragile rootless-Podman-on-a-starved-VM setup. This PR (app side): - runs-on: self-hosted -> xi2ix-website (the generic label let this job land on runners with no Docker capability) - DOCKER_HOST -> the DinD sidecar's plain Docker socket, with .toolchain/env.sh auto-detecting so local dev (rootless Podman) keeps working unchanged

forgeadmin added 1 commit

2026-07-12 14:58:19 +00:00

fix(ci): route to dedicated in-cluster runner, fix DOCKER_HOST for DinD

ci / gates (push) Failing after 1m15s

Details

ci / gates (pull_request) Failing after 1s

Details

a58fc79115

Root cause of the chronic CI outage tracked in
.planning/debug/ci-cd-tool-provisioning.md: the sole runner (VM 603) had
only 1.9GB RAM shared with infra-terraform's own CI, causing OOM/thrashing
under load -- matches every failure signature in that investigation
(multi-hour hangs, silent process deaths, an 11m38s `podman ps` query that
should return instantly). Live-diagnosed with direct runner-host access
this session.

Fix (infra side, terraform repo): a dedicated Forgejo Actions runner now
runs in-cluster on pve92's general-purpose k3s nodes (59GB free RAM
host-wide vs VM 603's 92MB), with a proper Docker-in-Docker sidecar instead
of the fragile rootless-Podman-on-a-starved-VM setup.

This commit (app side):
- ci.yaml / deploy.yaml: `runs-on: self-hosted` -> `runs-on: xi2ix-website`.
  The generic `self-hosted` label let this job land on ANY self-hosted
  runner, including ones with no Docker capability at all.
- ci.yaml / .toolchain/env.sh: DOCKER_HOST now points at the DinD sidecar's
  plain Docker socket (/var/run/docker.sock, bind-mounted into every job
  container) instead of a rootless-Podman per-UID path that no longer
  exists on the new runner. .toolchain/env.sh auto-detects which socket is
  present so local dev machines still running rootless Podman keep working
  unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>