diff --git a/docs/inventory.md b/docs/inventory.md index 91ebb0f..dd9ffc2 100644 --- a/docs/inventory.md +++ b/docs/inventory.md @@ -1,79 +1,76 @@ # Stack inventory -Source: `docker compose ls -a` on each host + `find /mnt/nas/home -maxdepth 2 -type d` (May 3, 2026). +State: May 3, 2026 — end of Tier 4. **Repo type** — see [strategy.md](strategy.md#repo-types). +**Image source** — `upstream` = published image used directly; `gitea-registry` = built off-host, pushed to `gitea.alexandru.macocian.me/amacocian/:`. -## Morgott (owner host) +## Migrated (`/mnt/nas/stacks//`) -| Old path (`home/`) | New path (`stacks/`) | Charlie repo | Repo type | +### Morgott (owner host) + +| Charlie repo | Stack path | Image source | Notes | |---|---|---|---| -| `caddy/` | `caddy/` | `caddy` | init | -| `authentik/authentik/` | `authentik/` | `authentik` | init | -| `open-ldap/` | `ldap/` | `ldap` | init | -| `ddclient/` | `ddclient/` | `ddclient` | init | -| `vaultwarden/` | `vaultwarden/` | `vaultwarden` | init | -| `bookstack/` | `bookstack/` | `bookstack` | init | -| `monica/monica/` | `monica/` | `monica` | init | -| `miniflux/` | `miniflux/` | `miniflux` | init | -| `mealie/mealie/docker/` | `mealie/` | `mealie` | init | -| `homarr/` | `homarr/` | `homarr` | init | -| `paperless-ngx/` | `paperless-ngx/` | `paperless-ngx` | init | -| `portainer/` | `portainer/` | `portainer` | init | -| `portainer-agent/` | `portainer-agent/` | `portainer-agent` | init | -| `otelcollector/` | `otelcollector/` | `otelcollector` | init | -| `VictoriaMetrics/deployment/docker/` | `victoriametrics/` | `victoriametrics` | init | -| `watchtower/` | `watchtower/` | `watchtower` | init (retire after Diun) | -| `Gssh/` | `gssh/` | `gssh` | rewrite | -| `ServerManagementUtils/` | `server-mgmt-utils/` | `server-mgmt-utils` | rewrite | +| `caddy` | `caddy/` | local build | edge reverse proxy + caddy-security | +| `authentik` | `authentik/` | upstream | IdP; postgres + redis + server + worker | +| `ldap` | `ldap/` | upstream | openldap + ldap-ui | +| `ddclient` | `ddclient/` | upstream | dynamic DNS | +| `vaultwarden` | `vaultwarden/` | upstream | password vault | +| `bookstack` | `bookstack/` | upstream | wiki (+ mariadb) | +| `monica` | `monica/` | gitea-registry `monica:4.x` | CRM (custom 4.x fork; + mariadb) | +| `miniflux` | `miniflux/` | upstream | RSS (+ postgres) | +| `mealie` | `mealie/` | upstream | recipes (sqlite) | +| `homarr` | `homarr/` | upstream | dashboard | +| `paperless-ngx` | `paperless-ngx/` | upstream | docs (+ redis) | +| `victoriametrics` | `victoriametrics/` | upstream | vm + vmauth + vmalert + alertmanager + grafana | +| `gssh` | `gssh/` | gitea-registry `gssh:latest` | web SSH terminal | -## Melina (owner host) +### Melina (owner host) -| Old path (`home/`) | New path (`stacks/`) | Charlie repo | Repo type | +| Charlie repo | Stack path | Image source | Notes | |---|---|---|---| -| `gitea/` | `gitea/` | `gitea` | init | -| `MediaServerCompose/` | `mediacompose/` | `mediacompose` | rewrite | -| `invidious/` | `invidious/` | `invidious` | init (was upstream-cloned; redo as our compose) | -| `searxng-docker/` | `searxng/` | `searxng` | init (same) | -| `planka/` | `planka/` | `planka` | init | -| `audiobookshelf/` | `audiobookshelf/` | `audiobookshelf` | init | -| `DirectoryAdmin/` | `directoryadmin/` | `directoryadmin` | rewrite | +| `gitea` | `gitea/` | upstream | the git server itself (deployed manually, never via runner-on-self) | +| `mediacompose` | `mediacompose/` | upstream | jellyfin + arr + qbt-via-gluetun | +| `searxng` | `searxng/` | upstream | metasearch (+ valkey) | +| `invidious` | `invidious/` | gitea-registry `invidious:oidc-support` | YouTube frontend (+ companion + db + gluetun) | +| `planka` | `planka/` | upstream | kanban (+ postgres) | +| `audiobookshelf` | `audiobookshelf/` | upstream | audiobooks | +| `directoryadmin` | `directoryadmin/` | gitea-registry `directoryadmin:latest` | LDAP admin UI | -## Mohg +## Pending (Tier 5 — long tail) -TBD — re-run inventory. +| Charlie repo | Old path | Owner | Repo type | Status | +|---|---|---|---|---| +| `portainer` | `home/portainer/` | morgott | init | not started | +| `portainer-agent` | `home/portainer-agent/` | morgott + melina (shared) | init | not started | +| `otelcollector` | `home/otelcollector/` | morgott + melina (shared) | init | not started | +| `watchtower` | `home/watchtower/` | morgott | init | not started — archive after Diun | +| `shell` | `home/shell/` | — (config, all hosts) | init | not started | +| `first-time-setup` | `home/first-time-setup.md` | — (docs) | init | not started | +| `server-mgmt-utils` | `home/ServerManagementUtils/` | morgott | rewrite | mostly dead — extract temp/storage scripts (move to `otelcollector/scripts/` or `host-scripts`), then archive | +| `plist` | `home/plist/` | ? | TBD | classify before deciding | -## Config-only (not deployed, but version-controlled) +## Mirrors (Gitea pull-mirror, no host action) -| Old path | New path | Charlie repo | +Naming: `Charlie/-mirror`. + +| Mirror | Upstream | Status | |---|---|---| -| `home/shell/` | `stacks/shell/` | `shell` | -| `home/first-time-setup.md` | `stacks/first-time-setup/` | `first-time-setup` | -| `home/plist/` | TBD — classify | — | +| `gssh` source | `https://github.com/AlexMacocian/Gssh` | pushed to `amacocian/Gssh` (manual) | +| `monica` source | `https://github.com/AlexMacocian/monica` (4.x) | mirror under `amacocian/monica` | +| `directoryadmin` source | `https://github.com/AlexMacocian/DirectoryAdmin` | mirror under `amacocian/DirectoryAdmin` | +| `invidious` source | `https://github.com/AlexMacocian/invidious` (oidc-support) | mirror under `amacocian/invidious` | -## Mirrors - -Pull-mirror only (Gitea config; no host action). Naming: `Charlie/-mirror`. - -| Mirror | Upstream | -|---|---| -| `invidious-mirror` | https://github.com/iv-org/invidious | -| `searxng-docker-mirror` | https://github.com/searxng/searxng-docker | -| `victoriametrics-mirror` | https://github.com/VictoriaMetrics/VictoriaMetrics | -| `mealie-mirror` | https://github.com/mealie-recipes/mealie | -| `paperless-ngx-mirror` | https://github.com/paperless-ngx/paperless-ngx | -| `nitter-mirror` | https://github.com/zedeus/nitter | -| `caddy-mirror` | https://github.com/caddyserver/caddy | -| `authentik-mirror` | https://github.com/goauthentik/authentik | -| `gitea-mirror` | https://github.com/go-gitea/gitea | -| `act-runner-mirror` | https://gitea.com/gitea/act_runner | +Optional later (upstreams we don't customise but want offline): +`caddy`, `authentik`, `paperless-ngx`, `mealie`, `victoriametrics`, `searxng-docker`, `nitter`, `gitea`, `act_runner`. ## Non-stack dirs | Path | Action | |---|---| | `/mnt/nas/ca/` | leave; out of scope | -| `/mnt/nas/ldap/` → `/mnt/nas/ldap-certs/` | rename during reorg | -| `/mnt/nas/media-server/` → `/mnt/nas/media/` | rename when `mediacompose` migrates | -| `/mnt/nas/portainer/data/` → `/mnt/nas/data/portainer/` | move when `portainer` migrates | -| `/mnt/nas/home/` | delete after every stack migrated | +| `/mnt/nas/ldap/` | rename → `/mnt/nas/ldap-certs/` (deferred — not blocking anything) | +| `/mnt/nas/media-server/` | rename → `/mnt/nas/media/` (deferred — bind paths in mediacompose `.env` reference the old name) | +| `/mnt/nas/portainer/data/` | move → `/mnt/nas/data/portainer/` when portainer migrates | +| `/mnt/nas/home/*.retired/` | delete after Tier 5 done and one week of stability | +| `/mnt/nas/home/` | rmdir after all `.retired/` dirs removed | diff --git a/docs/repos.md b/docs/repos.md index 7eaf241..5728913 100644 --- a/docs/repos.md +++ b/docs/repos.md @@ -1,74 +1,40 @@ # Repo plan — `Charlie/*` -Carving order. See [inventory.md](inventory.md) for the full list. Migration recipe: [migration.md](migration.md). Sequenced runbook: [reorg.md](reorg.md). +See [inventory.md](inventory.md) for the canonical state. Migration recipe: [migration.md](migration.md). -## Tier 0 — bootstrap +## Done -| Repo | Owner | Status | -|---|---|---| -| `project-charlie` | — | done | +### Tier 1 — pilot (Day 1) +`ddclient`, `homarr`, `vaultwarden` -## Tier 1 — pilot +### Tier 2 — our code (Day 1) +`caddy`, `gssh`, `directoryadmin`, `mediacompose` -Prove the loop end-to-end. Smallest blast radius. +### Tier 3 — pinned upstream + stateful (Day 1) +`ldap`, `authentik`, `bookstack`, `monica`, `miniflux`, `planka`, `audiobookshelf`, `gitea` -| Repo | Owner | Source | Type | +### Tier 4 — was upstream-cloned, collapsed to canonical (Day 1) +`mealie`, `paperless-ngx`, `victoriametrics`, `searxng`, `invidious` + +## Pending — Tier 5 + +Order: + +| # | Repo | Owner | Notes | |---|---|---|---| -| `ddclient` | morgott | `home/ddclient/` | init | -| `homarr` | morgott | `home/homarr/` | init | -| `vaultwarden` | morgott | `home/vaultwarden/` | init | +| 1 | `portainer` | morgott | mechanical | +| 2 | `portainer-agent` | shared | mechanical; one repo, deployed on every host | +| 3 | `watchtower` | morgott | trivial; will archive after Diun is up | +| 4 | `otelcollector` | shared | one compose, multi-host. Design decision: how do we host-pin `.env`? | +| 5 | `shell` | — (no docker) | clone target: `/usr/local/share/charlie-shell/` | +| 6 | `first-time-setup` | — (docs) | bootstrap docs / scripts | +| 7 | `server-mgmt-utils` | morgott | mostly dead. Extract temp/storage scripts to `otelcollector/scripts/` or new `host-scripts` repo, then archive the original. | +| ? | `plist` | TBD | classify before deciding | -## Tier 2 — our code +## Build pipelines (deferred) -| Repo | Owner | Source | Type | -|---|---|---|---| -| `gssh` | morgott | `home/Gssh/` | rewrite | -| `directoryadmin` | melina | `home/DirectoryAdmin/` | rewrite | -| `mediacompose` | melina | `home/MediaServerCompose/` | rewrite | -| `server-mgmt-utils` | morgott | `home/ServerManagementUtils/` | rewrite | -| `caddy` | morgott | `home/caddy/` (custom Dockerfile) | init | - -## Tier 3 — pinned upstream images, plain init - -| Repo | Owner | Source | -|---|---|---| -| `ldap` | morgott | `home/open-ldap/` | -| `authentik` | morgott | `home/authentik/authentik/` | -| `bookstack` | morgott | `home/bookstack/` | -| `monica` | morgott | `home/monica/monica/` | -| `miniflux` | morgott | `home/miniflux/` | -| `planka` | melina | `home/planka/` | -| `audiobookshelf` | melina | `home/audiobookshelf/` | -| `gitea` | melina | `home/gitea/` (deployed by morgott runner — never self) | - -## Tier 4 — was upstream-cloned, now plain init - -These previously lived as upstream-clones with `docker-compose.override.yml`. We're collapsing override → canonical compose. **Hand-merge step required** in [migration.md](migration.md). - -| Repo | Owner | Source | Image source | -|---|---|---|---| -| `invidious` | melina | `home/invidious/` | `quay.io/invidious/invidious` | -| `searxng` | melina | `home/searxng-docker/` | `searxng/searxng` | -| `victoriametrics` | morgott | `home/VictoriaMetrics/deployment/docker/` | `victoriametrics/victoria-metrics` | -| `mealie` | morgott | `home/mealie/mealie/docker/` | `ghcr.io/mealie-recipes/mealie` | -| `paperless-ngx` | morgott | `home/paperless-ngx/` | `ghcr.io/paperless-ngx/paperless-ngx` | - -## Tier 5 — long tail / utilities - -| Repo | Owner | Source | Type | -|---|---|---|---| -| `otelcollector` | shared | `home/otelcollector/` | init | -| `portainer` | morgott | `home/portainer/` | init | -| `portainer-agent` | shared | `home/portainer-agent/` | init | -| `shell` | — (config-only) | `home/shell/` | init | -| `first-time-setup` | — (docs-only) | `home/first-time-setup.md` | init | -| `watchtower` | morgott | `home/watchtower/` | init then archive | - -## Mirrors - -See [inventory.md](inventory.md#mirrors). Configured in Gitea UI; no host action. - -## Deferred - -`plist/` — classify before deciding. -`nitter/`, `vrising/` — on disk, not running. Leave as-is or archive. +Once runners are up, replace "build on laptop" with workflows on: +- `amacocian/Gssh` → push `gssh:latest` +- `amacocian/DirectoryAdmin` → push `directoryadmin:latest` +- `amacocian/monica` (`4.x`) → push `monica:4.x` +- `amacocian/invidious` (`oidc-support`) → push `invidious:oidc-support` diff --git a/docs/roadmap.md b/docs/roadmap.md index 3c51e88..8f2f0b9 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,22 +1,38 @@ # Roadmap -## Phase 1 — reorg + CI plumbing +## Phase 1 — reorg + Charlie repos - [x] Inventory `/mnt/nas/` stacks. - [x] Push `Charlie/project-charlie`. -- [ ] Snapshot `volume1/ubuntu` on miquella. -- [ ] Create empty `Charlie/` repos per [repos.md](repos.md). -- [ ] Create Gitea pull-mirrors per [inventory.md](inventory.md#mirrors). -- [ ] Walk [reorg.md](reorg.md) Tier 1 (pilot: `ddclient`). -- [ ] Walk Tiers 2–5. -- [ ] Delete `/mnt/nas/home/`. -- [ ] Design + deploy runner stack (`Charlie/runners`). +- [x] Snapshot `volume1/ubuntu` on miquella. +- [x] Tier 1 pilot (`ddclient`, `homarr`, `vaultwarden`). +- [x] Tier 2 (`caddy`, `gssh`, `directoryadmin`, `mediacompose`). +- [x] Tier 3 (`ldap`, `authentik`, `bookstack`, `monica`, `miniflux`, `planka`, `audiobookshelf`, `gitea`). +- [x] Tier 4 — collapse upstream-cloned stacks (`mealie`, `paperless-ngx`, `victoriametrics`, `searxng`, `invidious`). +- [ ] Tier 5 — long tail. See [repos.md](repos.md#pending--tier-5). +- [ ] Sweep `/mnt/nas/home/*.retired/` after stability check. +- [ ] Delete `/mnt/nas/home/` once empty. +- [ ] Mohg inventory + migration. + +## Phase 2 — CI + +- [ ] Design runner stack — `Charlie/runners`. Open question: secrets sourcing (vaultwarden? Gitea Actions secrets? SOPS?). +- [ ] Stand up runners on morgott + melina, register org-scope to `Charlie`. - [ ] First end-to-end pipeline: push to `Charlie/ddclient` → webhook → redeploy on morgott. +- [ ] Build pipelines on `amacocian/*` source repos to push images to gitea registry (replaces "build on laptop"). +- [ ] Per-stack `deploy.yaml` workflows in every `Charlie/`. -## Phase 2 — Automation +## Phase 3 — Automation -- Diun watching registries → webhook → `redeploy` workflow with `force-pull`. -- Retire watchtower. -- Drift detection: running compose vs repo compose. -- Backup verification against miquella. -- Cert/secret expiry checks (authentik, caddy, signing CA). +- [ ] Diun watching registries → webhook → `redeploy` workflow with `force-pull`. +- [ ] Retire `watchtower`. +- [ ] Drift detection: running compose vs repo compose. +- [ ] Backup verification against miquella. +- [ ] Cert/secret expiry checks (authentik, caddy, signing CA). +- [ ] Secret rotation pass — every stack with secrets exposed during reorg (most of them). + +## Phase 4 — NAS layout polish + +- [ ] Rename `/mnt/nas/media-server/` → `/mnt/nas/media/` and update mediacompose `.env`. +- [ ] Rename `/mnt/nas/ldap/` → `/mnt/nas/ldap-certs/`. +- [ ] Move `/mnt/nas/portainer/data/` → `/mnt/nas/data/portainer/` (when portainer migrates).