project-charlie/docs/roadmap.md

Roadmap

Phase 1 — reorg + Charlie repos

• Inventory /mnt/nas/ stacks.
• Push Charlie/project-charlie.
• Snapshot volume1/ubuntu on miquella.
• Tier 1 pilot (ddclient, homarr, vaultwarden).
• Tier 2 (caddy, gssh, directoryadmin, mediacompose).
• Tier 3 (ldap, authentik, bookstack, monica, miniflux, planka, audiobookshelf, gitea).
• Tier 4 — collapse upstream-cloned stacks (mealie, paperless-ngx, victoriametrics, searxng, invidious).
• Tier 5 — long tail. See repos.md.
• Sweep /mnt/nas/home/*.retired/ after stability check.
• Delete /mnt/nas/home/ once empty.
• Mohg inventory + migration.
• Malenia (mac) re-onboard. NFS mount currently unreliable from malenia → otelcollector mac install deferred. Investigate NFS/SMB/AFP mount options or switch to a different sync mechanism.

Phase 2 — CI

• Reusable deploy workflow + SOPS decrypt action shipped in Charlie/project-charlie. Design + caller guide: automation.md. Replaces per-stack deploy.sh.
• Stand up runners on morgott + melina, register org-scope to Charlie. Design: runners.md. Repo: Charlie/runners.
• First end-to-end pipeline: SOPS pilot on Charlie/ddclient per secrets.md → Pilot. Confirms the reusable workflow + runner + decrypt loop end-to-end.
• Roll the SOPS + reusable-workflow pattern across the rest of the migrated stacks, tier by tier. Drop each stack's deploy.sh as it migrates.
• Build pipelines on amacocian/* source repos to push images to gitea registry (replaces "build on laptop").
• Multi-host matrix workflow for shared stacks (otelcollector, portainer-agent).

Phase 3 — Automation

• Diun watching registries → webhook → redeploy workflow with force-pull (replaces the role watchtower used to play).
• Retire watchtower (decommissioned May 3, 2026 — dir at /mnt/nas/home/watchtower.retired/, no replacement yet).
• Drift detection: running compose vs repo compose.
• Backup verification against miquella.
• Cert/secret expiry checks (authentik, caddy, signing CA).
• Secret rotation pass — every stack with secrets exposed during reorg (most of them).

Phase 4 — NAS layout polish

• Rename /mnt/nas/media-server/ → /mnt/nas/media/ and update mediacompose .env.
• Rename /mnt/nas/ldap/ → /mnt/nas/ldap-certs/.
• Move /mnt/nas/portainer/data/ → /mnt/nas/data/portainer/ (when portainer migrates).

Phase 5 — Cross-cutting concerns (backlog)

These don't block earlier phases, but capture them now so they're not forgotten.

Centralization

• Centralize deploy.sh. Solved by the reusable deploy-stack.yml workflow + sops-decrypt action in this repo. Stack repos call it from .gitea/workflows/deploy.yaml; per-stack deploy.sh files are deleted on each stack's SOPS migration. See automation.md.
• Centralize shell setup. Charlie/shell repo with profile.d/, inputrc, etc., cloned to /usr/local/share/charlie-shell/ (or /etc/profile.d/charlie.sh sourcing a NAS path). Same delivery channel as charlie-tools.
• Centralize host-level scripts. sync-ca.sh, future temp/storage metric collectors (after server-mgmt-utils is dismantled), maybe a charlie-host-bootstrap to handle SSSD / docker / mounts. Likely lives under Charlie/charlie-tools or a sibling.

Observability

• Grafana dashboards as code. Grafana 12.2's UI Git Sync only supports github.com — Gitea isn't an option in the wizard. Pattern: dashboards live as JSON files inside Charlie/victoriametrics/dashboards/, mounted into grafana's provisioning path. Edits in the UI go to grafana's DB but UI shows a "Cannot save provisioned dashboard" dialog with a "Copy JSON" button — paste into the file in the repo, push, file-watcher in grafana picks it up. Sidecar that auto-syncs UI edits → git deferred until the manual loop becomes painful.

Secrets

• Decide on secrets sourcing for runners + stacks. Design + survey of existing tools in secrets.md. Open: bootstrap auth model, per-host vs per-collection scoping. Settle before any deploy pipeline that needs to write a .env.

Identity & access

• Extract SSH cert issuance into a standalone service. Charlie/cert-issuer (step-ca) deployed on morgott May 4, 2026. Gssh rewritten as a step-ca client; CA key no longer mounted into Gssh. Follow-up work (sync-ca rewrite, OIDC provisioner, host certs, TPM-sealed CA key) tracked in cert-issuer.md → Next steps.
• Auto-mount user home folders on SSH connect. Currently each user's $HOME is local to the host they SSH into. Want each user's home to be a NAS-backed share automounted on connect (NFS or autofs + LDAP user attribute). Should pair with the LDAP-backed SSSD setup (Charlie/ldap). Design before more users land.
• Miquella into the LDAP/SSH fold. DSM is currently outside the SSSD-backed identity. Configure DSM's LDAP client to point at Charlie/ldap, trust the SSH signing CA (/volume1/ubuntu/ca/ssh/ssh_user_ca.pub), and accept ephemeral certs from gssh. Goal: malenia + miquella both onboarded into the gssh terminal flow with LDAP-driven access.
• Per-host git identity. Each host needs its own credential to talk to Gitea — currently we paste a PAT inline. Decide: per-host PAT stored in the host's keyring, host-keyed SSH key registered as a deploy key on each Charlie/<stack> (rotation pain), or one "host-fleet" service account in Gitea with org-scope read + per-stack write.
• Per-repo permissions via LDAP-backed service accounts. Use Charlie/ldap to provision service accounts (e.g. svc-runner-morgott, svc-deploy-mediacompose) that Gitea recognises through OIDC. Per-repo read / write / admin roles map to LDAP groups. Replaces the current "everything pushes as amacocian" pattern. Required before runners go live so a compromised runner has bounded blast radius.

Cleanup

• Server-mgmt-utils. Mostly dead. Extract the temp/storage metric scripts into either Charlie/otelcollector/scripts/ (so otel can scrape them) or a new Charlie/host-scripts repo. Then archive the original.
• plist/ — classify and decide.