project-charlie/docs/strategy.md

Strategy

Pattern

GitOps via Gitea Actions runners. The runner is stateless and SSHes into the target host using a short-lived step-ca-issued cert. The job is a thin caller of the shared deploy-stack.yml reusable workflow defined in this repo — see automation.md.

dev box ── git push ──► Gitea (Charlie/<stack>) ── webhook ──► any runner
                                                                   │
                                                                   │ ssh sys-gitea-runner@<host>
                                                                   │ (step-ca cert, 5min)
                                                                   ▼
                                          on the target host:
                                            git fetch && git reset --hard origin/main
                                            sops --decrypt …  (if `decrypt:` set)
                                            docker compose pull && docker compose up -d
                                            rm decrypted plaintext

The git working tree at /mnt/nas/stacks/<stack>/ is the deployment. Runtime state lives separately under /mnt/nas/data/<stack>/ (see layout.md).

Rules

• One owner host per stack (singleton stacks). The host named in the workflow's host: input is where the runner SSHes.
• Shared stacks (run on multiple hosts — see Shared stacks) follow the same NFS-shared-tree model but each host runs its own docker compose up -d against host-specific overrides.
• No hand-edits on the host once a stack is repo-managed. The deploy workflow does git fetch && git reset --hard, which clobbers any drift on purpose.
• Runtime state never enters git. Compose files bind into /mnt/nas/data/<stack>/, not into the working tree.
• .env ignored, .env.example tracked. Plaintext secrets stay out — encrypted form lives under secrets/.
• Image tags pinned. No :latest.
• Any runner can deploy any stack to any host. Routing-by-runner-label is intentionally avoided (Gitea bug; see automation.md → Gitea quirks). The runner SSHes to wherever host: points.

Repo types

Three. (We dropped upstream+overrides — see Why no upstream cloning.)

init

Stack is currently just a NAS folder. git init in the new /mnt/nas/stacks/<stack>/, push to Charlie/<name>. Recipe: migration.md.

rewrite

Code we author, currently a clone of a github.com repo. Move to /mnt/nas/stacks/<stack>/, git remote set-url origin to Charlie. Recipe: migration.md.

mirror

Pull-mirror of an upstream we don't customise (or want as offline backup). Configured via Gitea's pull-mirror feature. Read-only, never deployed from. Naming: Charlie/<name>-mirror.

Shared stacks

Some stacks run on multiple hosts with the same image but slightly different configuration (e.g. otelcollector, portainer-agent). The pattern, borrowed from Ansible inventories + Kustomize overlays + Kubernetes DaemonSets:

Charlie/<stack>/
├── docker-compose.yml             # the "what" — default, used by hosts without an override
├── docker-compose.<host>.yml      # full per-host compose (when the delta is structural)
├── envs/
│   ├── morgott.env                # per-host env (Ansible host_vars equivalent)
│   ├── melina.env
│   ├── mohg.env
│   └── miquella.env
├── overrides/                     # additive overlay deltas (when env can't express it)
│   └── <host>.yml
├── secrets/                       # SOPS-encrypted; decrypted at deploy time
│   └── env.sops.yaml              # (or per-host: <host>.env.sops.yaml)
├── .sops.yaml                     # recipients (per secrets.md)
├── .gitea/workflows/deploy.yaml   # calls the reusable workflow in this repo
└── README.md

Per-host resolution order

Applied identically by deploy-stack.yml and by the manual fallback recipe:

1. If docker-compose.<host>.yml exists → use only that (full per-host compose).
2. Else docker-compose.yml + (if present) overrides/<host>.yml layered on top.
3. If envs/<host>.env exists → loaded with --env-file in either case.

Hostname is lowercased (hostname -s | tr '[:upper:]' '[:lower:]'). The reusable workflow takes the host from its host: input, which the caller sets explicitly (often via a matrix).

When to use which

Delta type	Use
Different scalar value (port, env var, URL)	envs/<host>.env
Add an extra service or scalar field	overrides/<host>.yml (compose merges additively)
Replace a list (volumes, command, networks)	docker-compose.<host>.yml (full file)
Different image	docker-compose.<host>.yml

Compose's !override and !reset YAML tags exist for list-replacement, but support is uneven across compose versions (notably broken on Synology DSM's bundled compose). Prefer the full-file approach when replacing lists.

Canonical deploy workflow

Every stack repo carries this in .gitea/workflows/deploy.yaml (no deploy.sh):

name: Deploy
on:
  push:
    branches: [main]
  workflow_dispatch:

jobs:
  deploy:
    uses: https://sys-gitea-runner:${{ secrets.RUNNER_REPO_PAT }}@gitea.alexandru.macocian.me/Charlie/project-charlie/.gitea/workflows/deploy-stack.yml@main
    with:
      host: morgott
      decrypt: |
        secrets/env.sops.yaml -> .env
    secrets: inherit

Drop the decrypt: for stacks without secrets. Use a matrix for shared stacks. Full input reference + manual-fallback recipe: automation.md. The token-prefix uses: URL is a Gitea workaround documented in automation.md → Gitea quirks.

Mental model

• "What" = docker-compose.yml (or per-host file).
• "Where" = the host named in the workflow's host: input. Any runner picks up the job; the runner SSHes into that host and runs everything there.
• Per-host vars = envs/<host>.env.
• Per-host structural deltas = overrides/<host>.yml for additive, docker-compose.<host>.yml for replacement.
• Group defaults = compose env-var defaults (${VAR:-default}).

Shared-stack workflow shape

For stacks that fan out across hosts, the matrix lives in the caller and the reusable workflow handles per-host file resolution per the rules above:

jobs:
  deploy:
    strategy:
      matrix:
        host: [morgott, melina, mohg, miquella]
    uses: https://sys-gitea-runner:${{ secrets.RUNNER_REPO_PAT }}@gitea.alexandru.macocian.me/Charlie/project-charlie/.gitea/workflows/deploy-stack.yml@main
    with:
      host: ${{ matrix.host }}
      decrypt: |
        secrets/${{ matrix.host }}.env.sops.yaml -> envs/${{ matrix.host }}.env
    secrets: inherit

That's a Kubernetes DaemonSet shape adapted to compose + Gitea Actions.

Why no upstream cloning

Previously we considered cloning upstream's source on the host and layering a docker-compose.override.yml. Dropped because:

• For every "upstream image" stack (invidious, searxng, victoriametrics, mealie, paperless-ngx), we only ever ran the published image. Their source tree is dead weight on the host.
• Override-on-template means the upstream's compose can change under us silently.
• Two-remote / rebase / force-push dance was the price for dubious value.

Instead: our docker-compose.yml references the published image directly. It is the only compose. Upstream's compose is a reference document we read in their mirror repo when we want to see what changed.

Image-update flow (Phase 3)

Diun watches registries; on a tag/digest change it POSTs to a Gitea webhook that triggers a redeploy workflow with force-pull: true on the owner host. Replaces watchtower.

Secrets

Decided. SOPS-in-repo with one age key per host. The reusable deploy-stack.yml workflow SSHes into the target host as sys-gitea-runner and runs sops --decrypt there using the host's /etc/charlie/age.key. Plaintext stays on the target host (mode 0600), gets cleaned up by an if: always() step. Full design and rollout plan: secrets.md.

Why not Portainer/Komodo/Dockge

• Portainer Stacks-from-Git: works, but no submodule support, opinionated about working-tree location, adds a control plane on top of git.
• Komodo: nice, also pull-based, also relocates working trees. Reconsider if we ever want a UI.
• Dockge: file mgmt only, no git wiring.

The runner-driven pattern keeps us in plain git + docker compose and adds nothing the team doesn't already understand.