project-charlie/docs/roadmap.md

Roadmap

Open work, grouped by theme. Done items are dropped — git history has them.

Hosts

• Mohg reonboarding. Wipe + reinstall mohg, run a charlie-host-bootstrap (see Centralization) end-to-end. Folds mohg into auto-deploy for portainer-agent and otelcollector.
• Malenia (mac) re-onboard. NFS mount unreliable from malenia → otelcollector mac install deferred. Investigate NFS/SMB/AFP options or switch sync mechanism. Once reonboarded, refactor otelcol.mac.yaml to read ${env:VM_INGEST_AUTH} and drop the credential from git (collapses rotation from 3 places to 1).
• Break the LDAP-on-morgott boot cycle. ldap runs as a docker stack on morgott, so sssd can't be a hard prereq for docker (sssd → ldap container → docker → circular). We currently work around it with a local /etc/group shadow entry for charlie-deploy (see onboarding.md → Charlie deploy group, step 3). Architecturally cleaner: move Charlie/ldap onto a host whose docker socket doesn't depend on an LDAP-resolved group, or run slapd natively somewhere. Removes the GID-duplication and the "if LDAP renumbers the group, update each host's /etc/group" footgun.
• Miquella into the LDAP/SSH fold. DSM is currently outside SSSD-backed identity. Configure DSM's LDAP client to point at Charlie/ldap, trust the SSH signing CA, accept ephemeral certs from gssh. Goal: malenia + miquella both onboarded into the gssh terminal flow with LDAP-driven access.

Automation / observability

• Diun watching registries → webhook → redeploy workflow with force-pull (replaces what watchtower used to do).
• Drift detection. Running compose vs repo compose.
• Backup verification against miquella.
• Cert/secret expiry checks (authentik, caddy, signing CA).
• Build pipelines on amacocian/* source repos to push images to gitea registry (replaces "build on laptop").
• Grafana dashboards as code. Grafana 12.2's UI Git Sync only supports github.com — Gitea isn't an option. Pattern: dashboards live as JSON files inside Charlie/victoriametrics/dashboards/, mounted into grafana's provisioning path with foldersFromFilesStructure: true so subdirs become Grafana folders. Edits flow via a "Cannot save provisioned dashboard" → "Copy JSON" → paste into repo loop. Sidecar that auto-syncs UI edits → git deferred until manual loop becomes painful.
• Initial dashboard set. hosts/ (overview, host-detail, thermal-power, filesystem-network), processes/ (overview + CPU + memory broken down per process), containers/ (overview, per-container resources, per-stack roll-up). Container metrics added by enabling the otelcol docker_stats receiver (mounts /var/run/docker.sock, promotes com.docker.compose.project|service to metric labels).
• Container CVE sweep. Charlie/trivy-server (morgott, central vuln-DB cache + daily compose-declared image sweep) + Charlie/trivy-runner (per-host, daily docker ps sweep). Results land as Prometheus textfiles → node-exporter (textfile collector) → otelcol → VM. Dashboards under containers/security/. LAN-only between the two stacks; no public DNS, no auth — morgott + melina sit on the secure intranet.
• Trivy CVE alerting. vmalert rule on trivy_image_vulnerabilities{severity="CRITICAL"} > 0 → alertmanager via the existing route. Deferred until the sweep has produced a stable baseline (otherwise the first run is one giant page).
• CI build-time CVE scan. Run trivy image as a step inside the eventual amacocian/* build pipelines (see "Build pipelines" above) so we catch HIGH/CRITICAL at build time, not just at the next nightly sweep.
• Runner observability (phase 2). Gitea Actions runner dashboards in runners/. Likely a small exporter polling Gitea's Actions API → otelcol → VM. Tracked in Charlie/victoriametrics/dashboards/runners/README.md.

Deploy workflow

• Per-host decrypt path in deploy-stack.yml. Today both halves of a shared-stack deploy decrypt to the same /mnt/nas/stacks/<stack>/.env over NFS, so we serialize via needs: instead of strategy.matrix. If decrypt landed at e.g. /run/charlie-secrets/<stack>-<host>/.env (host-local tmpfs, per-host filename), matrix-with-true-parallelism becomes safe and the workflow files for shared stacks collapse back to a single matrix job. Also helps once Gitea's runs-on: label routing in reusable workflows is fixed (see #32348).
• Tag pinning for cross-repo uses:. Caller workflows reference @main, which means after every push to this repo or the runner image, runners need their act cache busted (sudo docker exec runner rm -rf /root/.cache/act). Switch to @v1 once contract stabilises.

Centralization

• Charlie/shell. profile.d/, inputrc, etc., cloned to /usr/local/share/charlie-shell/ (or /etc/profile.d/charlie.sh sourcing a NAS path).
• Charlie/charlie-tools (or similar). Host-level scripts: sync-ca.sh, future temp/storage metric collectors (after server-mgmt-utils is dismantled).
• charlie-host-bootstrap. Script (or Ansible playbook) that runs every numbered section of onboarding.md from a fresh OS install. Validation plan: wipe mohg, run the bootstrapper end-to-end, confirm it joins the fleet without manual fixups. Pairs with Mohg reonboarding.

Identity & access

• Auto-mount user home folders on SSH connect. Currently each user's $HOME is local to the host they SSH into. Want each user's home to be a NAS-backed share automounted on connect (NFS or autofs + LDAP user attribute). Should pair with the LDAP-backed SSSD setup (Charlie/ldap).
• Per-host git identity. Each host needs its own credential to talk to Gitea — currently we paste a PAT inline. Decide: per-host PAT in the host's keyring, host-keyed SSH key registered as a deploy key on each Charlie/<stack> (rotation pain), or one "host-fleet" service account with org-scope read + per-stack write.
• Per-repo permissions via LDAP-backed service accounts. Use Charlie/ldap to provision service accounts (e.g. svc-runner-morgott, svc-deploy-mediacompose) that Gitea recognises through OIDC. Per-repo read / write / admin roles map to LDAP groups. Replaces the "everything pushes as amacocian" pattern. Reduces blast radius of a compromised runner.
• Cert-issuer follow-ups — see cert-issuer.md → Pending (X.509 root rollout, Authentik OIDC provisioner, SSH host certs, TPM-sealed CA key, sync-ca.sh rewrite).

Cleanup

• NAS layout polish.
  • Rename /mnt/nas/media-server/ → /mnt/nas/media/ and update mediacompose .env.
  • Rename /mnt/nas/ldap/ → /mnt/nas/ldap-certs/.
  • Move /mnt/nas/portainer/data/ → /mnt/nas/data/portainer/.
• server-mgmt-utils. Mostly dead. Extract the temp/storage metric scripts into either Charlie/otelcollector/scripts/ or new Charlie/host-scripts, then archive.
• plist/. Classify and decide.
• Secret rotation pass. Every stack with secrets exposed during the original NAS-to-git reorg.