project-charlie/docs/runners.md

Runners

Design for Charlie/runners — the Gitea Actions runner stack. This is the prerequisite that unblocks automation.md (the reusable deploy-stack.yml workflow) and the SOPS rollout.

Status: not yet stood up. Repo will be created when we start work.

Goal

Two self-hosted Gitea Actions runners — one on morgott, one on melina — registered org-scope to Charlie. Together they can deploy any singleton stack on either of those hosts and execute matrix jobs across both. We add a third (mohg) when the first stack on mohg moves to runner deploys.

Why two, why these two

• Cross-host self-update. Per conventions.md, a runner never redeploys itself mid-job. Morgott's runner deploys melina (incl. melina's runner); melina's runner deploys morgott (incl. morgott's runner). Two is the minimum that makes that loop work.
• Coverage. Together, morgott + melina own every Tier 1–4 stack (inventory.md). Mohg + miquella + malenia get runners later, on demand.

Image and orchestration

• Upstream gitea/act_runner image, pinned tag (no :latest).
• Per strategy.md → Shared stacks: single repo, per-host envs and (if needed) a per-host compose file. Runner labels derive from hostname -s | tr '[:upper:]' '[:lower:]' so workflow runs-on: [self-hosted, <host>] matches.
• Bind-mounts /var/run/docker.sock so jobs can docker compose against the host daemon. The runner is therefore as privileged as root on its host — same blast-radius assumption as everything else here.

Sketch — Charlie/runners/

Charlie/runners/
├── docker-compose.yml             # base
├── envs/
│   ├── morgott.env                # GITEA_INSTANCE_URL, RUNNER_LABELS, etc.
│   └── melina.env
├── secrets/
│   ├── morgott.token.sops.yaml    # registration token, encrypted to morgott
│   └── melina.token.sops.yaml     # ditto, encrypted to melina
├── .sops.yaml
└── .gitea/workflows/deploy.yaml   # uses Charlie/project-charlie deploy-stack.yml

The runner's labels (set via RUNNER_LABELS or config.yaml) must include the host's lowercase short hostname (e.g. morgott) plus self-hosted so the deploy workflow's runs-on: [self-hosted, "${{ inputs.host }}"] matches.

Bootstrap order

Bootstrapping is a chicken-and-egg dance because the deploy workflow depends on the runner that deploys it. One-time manual sequence:

1. Generate registration tokens. In the Gitea UI: site admin → Charlie org → Runners → "Create new Runner" twice (one per host). Note the tokens.

2. SOPS-encrypt the tokens into secrets/<host>.token.sops.yaml, recipients = host key + admin key (per secrets.md).

3. Manually deploy each runner — same steps the workflow would do, run by hand once. Recipe: automation.md → Manual fallback, targeting Charlie/runners.

4. Confirm both runners show as Idle in the Gitea UI.

5. From now on, runner updates flow through Charlie/runners's deploy workflow — but morgott's runner deploys melina's, and vice versa (cross-host rule). The matrix job should be host-scoped to the other host, not ${{ matrix.host }}-style. Concretely:

   jobs:
     deploy-melina:
       uses: Charlie/project-charlie/.gitea/workflows/deploy-stack.yml@main
       with:
         host: morgott        # morgott's runner deploys melina's stack
         stack-path: /mnt/nas/stacks/runners
         compose-args: --remove-orphans
       # ...but the compose this runs only brings up melina's container.
       # Per-host compose selection is via docker-compose.melina.yml.
   

   The cleanest shape is probably two workflows in Charlie/runners: deploy-morgott.yaml (run on melina's runner) and deploy-melina.yaml (run on morgott's runner). Decide once we get there.

Permissions model

Open question, captured here so it's visible. Final answer goes into secrets.md (or its successor) and into Charlie/runners's README once the stack exists.

• Runner Gitea identity. act_runner authenticates with a registration token at first run, then uses a session credential. Org-scope means the runner can read every Charlie/<stack> repo (which is what we want for cross-repo uses: Charlie/project-charlie/...@main). It can also see every Charlie/<stack>'s .gitea/workflows/. This is fine because source-of-truth is git, not the runner.
• Runner sudo. The runner's host user (alex or a dedicated gitea-runner) needs passwordless sudo for at least: git, docker compose, and rm inside /mnt/nas/stacks/. This is enforced by sudoers.d/ on the host. Per automation.md → Host prerequisites.
• Per-stack write access. Currently every push to Charlie/<stack> is by amacocian. The longer-term plan (roadmap.md → Per-repo permissions via LDAP-backed service accounts) is to issue per-stack service accounts via LDAP-backed OIDC. Ship runners first; tighten access after.

Open questions

1. One runner per host or many? Single runner with concurrency >1 is simpler. Multiple runners per host (each with a unique label) only if we want strict isolation between e.g. "deploy" and "build" jobs. v1: one per host.
2. Where does the runner's working directory live? Default /var/lib/gitea-runner/ (host-local), bind-mounted into the container. Runner caches jobs there. Don't put it on /mnt/nas — NFS + many small files is asking for trouble.
3. Job log retention. Gitea stores logs server-side; runner just streams. No host-side action needed.
4. Self-update for the runner image. Will fold into the diun-driven Phase 3 image-update flow once that exists. Until then: pin tag, bump by hand.

Out of scope here

• The actual Charlie/runners repo contents — written when we create the repo. This doc is the design that goes in.
• Build pipelines on amacocian/* repos (roadmap.md → Phase 2). Same runner pool, but workflows live elsewhere.