Charlie/project-charlie
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Reorg plan: stacks/+data/ layout, drop upstream+overrides, add layout/reorg docs

Browse Source
This commit is contained in:
Alexandru Victor Macocian
2026-05-03 13:50:13 +02:00
parent 615575b087
commit 88ba6f853a
8 changed files with 385 additions and 180 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+2
View File
@@ -9,9 +9,11 @@ Centralized source of truth for the homelab. Per-app stacks live as repos under
- [Identity & access](docs/identity.md)
- [Strategy](docs/strategy.md)
- [Conventions](docs/conventions.md)
- [Layout](docs/layout.md)
- [Stack inventory](docs/inventory.md)
- [Repo plan (`Charlie/*`)](docs/repos.md)
- [Migration recipe](docs/migration.md)
- [Reorg runbook](docs/reorg.md)
- [Roadmap](docs/roadmap.md)
## Assumptions
docs/inventory.md
+59 -52
View File
@@ -1,72 +1,79 @@
# Stack inventory
Source: `docker compose ls -a` on each host + `find /mnt/nas -name 'docker-compose.y*ml'`. Compose paths under `/mnt/nas/home/` unless noted.
Source: `docker compose ls -a` on each host + `find /mnt/nas/home -maxdepth 2 -type d` (May 3, 2026).
**Repo type:**
- `init` — no git history yet; `git init` in place, push to `Charlie/<name>`.
- `rewrite` — existing git repo (typically on github.com); change `origin` to `Charlie/<name>`.
- `upstream+overrides` — upstream repo we don't author; track `upstream`, add `origin` = `Charlie/<name>` containing only overrides.
- `mirror` — pull-mirror of an upstream repo we use unmodified, kept under `Charlie/<name>` for backup/availability.
**Repo type** — see [strategy.md](strategy.md#repo-types).
## Morgott (owner host)
| Stack | Path | Status | Repo type | Charlie repo |
|-------|------|--------|-----------|--------------|
| caddy | `caddy/` | running | init | `Charlie/caddy` |
| authentik | `authentik/authentik/` | running | init (overrides only) | `Charlie/authentik` |
| open-ldap | `open-ldap/` | running | init | `Charlie/ldap` |
| ddclient | `ddclient/` | running | init | `Charlie/ddclient` |
| vaultwarden | `vaultwarden/` | running | init | `Charlie/vaultwarden` |
| bookstack | `bookstack/` | running | init | `Charlie/bookstack` |
| monica | `monica/monica/` | running | init | `Charlie/monica` |
| miniflux | `miniflux/` | running | init | `Charlie/miniflux` |
| mealie | `mealie/mealie/docker/` | running | upstream+overrides | `Charlie/mealie` |
| homarr | `homarr/` | running | init | `Charlie/homarr` |
| paperless-ngx | `paperless-ngx/` | running | upstream+overrides | `Charlie/paperless-ngx` |
| portainer | `portainer/` | running | init | `Charlie/portainer` |
| portainer-agent | `portainer-agent/` | running | init | `Charlie/portainer-agent` |
| otelcollector | `otelcollector/` | running | init | `Charlie/otelcollector` |
| VictoriaMetrics | `VictoriaMetrics/deployment/docker/` | running (`docker`) | upstream+overrides | `Charlie/victoriametrics` |
| watchtower | `watchtower/` | running | init | `Charlie/watchtower` (retire after pipelines) |
| Gssh | `Gssh/` | running | rewrite | `Charlie/gssh` |
| Old path (`home/`) | New path (`stacks/`) | Charlie repo | Repo type |
|---|---|---|---|
| `caddy/` | `caddy/` | `caddy` | init |
| `authentik/authentik/` | `authentik/` | `authentik` | init |
| `open-ldap/` | `ldap/` | `ldap` | init |
| `ddclient/` | `ddclient/` | `ddclient` | init |
| `vaultwarden/` | `vaultwarden/` | `vaultwarden` | init |
| `bookstack/` | `bookstack/` | `bookstack` | init |
| `monica/monica/` | `monica/` | `monica` | init |
| `miniflux/` | `miniflux/` | `miniflux` | init |
| `mealie/mealie/docker/` | `mealie/` | `mealie` | init |
| `homarr/` | `homarr/` | `homarr` | init |
| `paperless-ngx/` | `paperless-ngx/` | `paperless-ngx` | init |
| `portainer/` | `portainer/` | `portainer` | init |
| `portainer-agent/` | `portainer-agent/` | `portainer-agent` | init |
| `otelcollector/` | `otelcollector/` | `otelcollector` | init |
| `VictoriaMetrics/deployment/docker/` | `victoriametrics/` | `victoriametrics` | init |
| `watchtower/` | `watchtower/` | `watchtower` | init (retire after Diun) |
| `Gssh/` | `gssh/` | `gssh` | rewrite |
| `ServerManagementUtils/` | `server-mgmt-utils/` | `server-mgmt-utils` | rewrite |
## Melina (owner host)
| Stack | Path | Status | Repo type | Charlie repo |
|-------|------|--------|-----------|--------------|
| gitea | `gitea/` | running | init | `Charlie/gitea` |
| MediaServerCompose | `MediaServerCompose/` | running | rewrite | `Charlie/mediacompose` |
| invidious | `invidious/` | running | upstream+overrides | `Charlie/invidious` |
| searxng-docker | `searxng-docker/` | running | upstream+overrides | `Charlie/searxng` |
| planka | `planka/` | running | init | `Charlie/planka` |
| audiobookshelf | `audiobookshelf/` | running | init | `Charlie/audiobookshelf` |
| DirectoryAdmin | `DirectoryAdmin/` | running | rewrite | `Charlie/directoryadmin` |
| portainer-agent | `portainer-agent/` | running | shared with morgott | `Charlie/portainer-agent` |
| otelcollector | `otelcollector/` | running | shared with morgott | `Charlie/otelcollector` |
| Old path (`home/`) | New path (`stacks/`) | Charlie repo | Repo type |
|---|---|---|---|
| `gitea/` | `gitea/` | `gitea` | init |
| `MediaServerCompose/` | `mediacompose/` | `mediacompose` | rewrite |
| `invidious/` | `invidious/` | `invidious` | init (was upstream-cloned; redo as our compose) |
| `searxng-docker/` | `searxng/` | `searxng` | init (same) |
| `planka/` | `planka/` | `planka` | init |
| `audiobookshelf/` | `audiobookshelf/` | `audiobookshelf` | init |
| `DirectoryAdmin/` | `directoryadmin/` | `directoryadmin` | rewrite |
## Mohg
TBD — re-run inventory.
## Non-stack dirs (kept on NAS, not stack repos)
## Config-only (not deployed, but version-controlled)
| Path | Purpose | Action |
|------|---------|--------|
| `/mnt/nas/ca/` | host CA material (ssh, ldap.lan) | leave; consider `Charlie/ca-tooling` for `sync-ca.sh` only |
| `/mnt/nas/ldap/certs/` | LDAP server certs | leave |
| `/mnt/nas/media-server/` | bulk media (downloads, movies, tv) | leave; bind target |
| `/mnt/nas/portainer/data/` | portainer state | leave |
| `/mnt/nas/home/shell/` | shared shell config (`profile.d`, `inputrc`) | `Charlie/shell` (init) |
| `/mnt/nas/home/plist/` | unclear | classify before action |
| `/mnt/nas/home/ServerManagementUtils/` | filebeat / metricbeat / logstash / orchestrator | rewrite → `Charlie/server-mgmt-utils` |
| Old path | New path | Charlie repo |
|---|---|---|
| `home/shell/` | `stacks/shell/` | `shell` |
| `home/first-time-setup.md` | `stacks/first-time-setup/` | `first-time-setup` |
| `home/plist/` | TBD — classify | — |
## Mirror-only (Charlie hosts a backup mirror, no overrides)
## Mirrors
Useful for upstreams we may patch later or want offline copies of:
- `invidious`, `searxng-docker`, `VictoriaMetrics`, `mealie`, `paperless-ngx`, `nitter`.
Pull-mirror only (Gitea config; no host action). Naming: `Charlie/<name>-mirror`.
(See [strategy.md](strategy.md) on Gitea pull-mirror config.)
| Mirror | Upstream |
|---|---|
| `invidious-mirror` | https://github.com/iv-org/invidious |
| `searxng-docker-mirror` | https://github.com/searxng/searxng-docker |
| `victoriametrics-mirror` | https://github.com/VictoriaMetrics/VictoriaMetrics |
| `mealie-mirror` | https://github.com/mealie-recipes/mealie |
| `paperless-ngx-mirror` | https://github.com/paperless-ngx/paperless-ngx |
| `nitter-mirror` | https://github.com/zedeus/nitter |
| `caddy-mirror` | https://github.com/caddyserver/caddy |
| `authentik-mirror` | https://github.com/goauthentik/authentik |
| `gitea-mirror` | https://github.com/go-gitea/gitea |
| `act-runner-mirror` | https://gitea.com/gitea/act_runner |
## Removed since last scan
## Non-stack dirs
`apache-guacamole`, `docker-dnsmasq`, `Docker-Minecraft-Paper`, `elasticsearch/`, `elk/`, `ferdium-server`, `LibreChat`, `localai`, `nitter` (still on disk, not running), `openobserve`, `reranker`, `scraper`, `step-ca`, `vrising` (game data only), `xstarbound`. `nitter` and `vrising` directories remain on disk but no longer have running stacks.
| Path | Action |
|---|---|
| `/mnt/nas/ca/` | leave; out of scope |
| `/mnt/nas/ldap/` → `/mnt/nas/ldap-certs/` | rename during reorg |
| `/mnt/nas/media-server/` → `/mnt/nas/media/` | rename when `mediacompose` migrates |
| `/mnt/nas/portainer/data/` → `/mnt/nas/data/portainer/` | move when `portainer` migrates |
| `/mnt/nas/home/` | delete after every stack migrated |
docs/layout.md
+80
View File
@@ -0,0 +1,80 @@
# Layout
Canonical `/mnt/nas/` shape after the reorg.
```
/mnt/nas/
├── stacks/ ← per-stack git working trees. Small, all-in-git.
│ ├── caddy/
│ ├── ldap/
│ ├── ...
├── data/ ← per-stack runtime state. Bind-mounted into containers.
│ ├── caddy/
│ ├── ldap/
│ ├── mediacompose/
│ │ ├── jellyfin/
│ │ ├── sonarr/
│ │ └── ...
│ ├── ...
├── media/ ← bulk media (renamed from media-server/)
│ ├── downloads/
│ ├── movies/
│ └── tv/
├── ca/ ← host CA material; out of scope for the reorg
├── ldap-certs/ ← LDAP server certs (renamed from ldap/); out of scope
└── home/ ← legacy; deleted at end of reorg
```
## Naming
- All directories under `stacks/` and `data/` are **kebab-case, lowercase**.
- Repo names under `Charlie/` match the `stacks/` directory name 1:1.
| Old | New |
|-----|-----|
| `home/Gssh/` | `stacks/gssh/` |
| `home/MediaServerCompose/` | `stacks/mediacompose/` |
| `home/DirectoryAdmin/` | `stacks/directoryadmin/` |
| `home/open-ldap/` | `stacks/ldap/` |
| `home/VictoriaMetrics/deployment/docker/` | `stacks/victoriametrics/` |
| `home/mealie/mealie/docker/` | `stacks/mealie/` |
| `home/authentik/authentik/` | `stacks/authentik/` |
| `home/searxng-docker/` | `stacks/searxng/` |
| `home/paperless-ngx/` | `stacks/paperless-ngx/` |
| `home/ServerManagementUtils/` | `stacks/server-mgmt-utils/` |
| `home/portainer-agent/` | `stacks/portainer-agent/` |
## Bind-mount convention
Every compose file binds **only** `/mnt/nas/data/<stack>/...` for runtime state. No bind into `./` (which is the git working tree).
```yaml
# stacks/ldap/docker-compose.yml
services:
openldap:
volumes:
- /mnt/nas/data/ldap/db:/var/lib/ldap
- /mnt/nas/data/ldap/config:/etc/ldap/slapd.d
```
Small read-only configs that *should* be version-controlled (Caddyfile, openldap bootstrap LDIFs, etc.) live inside the `stacks/<stack>/` working tree and are bind-mounted as files:
```yaml
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
```
## Out-of-scope dirs (leave alone for now)
- `/mnt/nas/ca/` — host CA material. Considered separately later (`Charlie/ca-tooling`).
- `/mnt/nas/ldap-certs/` — LDAP server certs (after rename from `ldap/`). Same.
- `/mnt/nas/#recycle` — Synology recycle bin.
## Stacks-only-as-config (no container)
Some `stacks/<name>/` may not deploy a container — they're just version-controlled config that belongs to the homelab:
- `stacks/shell/` — shared `profile.d`, `inputrc`. Cloned by hosts to `/usr/local/share/charlie-shell/`, sourced by `/etc/profile`.
- `stacks/first-time-setup/` — bootstrap docs / scripts (currently `home/first-time-setup.md`).
These have no `docker-compose.yml` and no entry in the runner deploy list.
docs/migration.md
+95 -61
View File
@@ -1,85 +1,119 @@
# Migration — NAS folder → `Charlie/<stack>` repo
# Migration
For `init`-type stacks. Run on the owner host. Container keeps running throughout.
Per-stack recipe to move `home/<old>/` → `stacks/<new>/` + `data/<new>/` and create `Charlie/<new>`.
## Pre-conditions
- Owner host SSH session as admin.
- Container is **stopped**: `cd /mnt/nas/home/<old> && sudo docker compose down`.
- Empty Gitea repo created (Charlie org, private, uninitialized).
## `init` — no existing git history
```bash
# 0. In Gitea UI: create empty Charlie/<name>, private, uninitialized.
OLD=/mnt/nas/home/<old>
NEW=/mnt/nas/stacks/<new>
DATA=/mnt/nas/data/<new>
cd /mnt/nas/home/<stack>
ls -la && du -sh */ 2>/dev/null # spot the heavy state dirs
# 1. Make destinations
sudo mkdir -p "$NEW" "$DATA"
sudo chown -R alex:alex "$NEW"
git init -b main
# 2. Move runtime state OUT of the old dir, into data/
# (Identify state dirs first: data/, db-data/, appdata/, cache/, …)
sudo mv "$OLD"/data "$DATA"/ 2>/dev/null || true
sudo mv "$OLD"/db-data "$DATA"/ 2>/dev/null || true
sudo mv "$OLD"/<other-state> "$DATA"/ 2>/dev/null || true
# .gitignore BEFORE first add — never stage runtime state
$EDITOR .gitignore # see template below
# 3. Copy the deployment files into the new working tree
cp "$OLD"/docker-compose.yml "$NEW"/
cp "$OLD"/docker-compose.override.yml "$NEW"/ 2>/dev/null || true
cp "$OLD"/.env "$NEW"/ 2>/dev/null || true
cp -r "$OLD"/<small-configs> "$NEW"/ 2>/dev/null || true
cp .env .env.example
$EDITOR .env.example # redact every secret
cd "$NEW"
git add .
git status
git diff --cached --stat
# If state dirs slipped through: fix .gitignore, `git rm --cached -r <dir>`, repeat.
# 4. Rewrite the compose so binds reference /mnt/nas/data/<new>/...
$EDITOR docker-compose.yml docker-compose.override.yml
git commit -m "Initial import from /mnt/nas/home/<stack>"
git remote add origin https://gitea.alexandru.macocian.me/Charlie/<name>.git
git push -u origin main
```
## `.gitignore` template
```gitignore
# runtime state
data/
db-data/
appdata/
*-data/
cache/
logs/
*.log
*.sqlite*
# secrets
# 5. .gitignore + .env.example
cat > .gitignore <<'EOF'
.env
.env.*
!.env.example
```
EOF
cp .env .env.example
$EDITOR .env.example # redact secrets
Tune per stack — e.g. `paperless-ngx` also has `consume/`, `export/`, `media/`, `media-bk/`.
## `rewrite`-type
```bash
cd /mnt/nas/home/<stack>
git remote rename origin github 2>/dev/null || true
git remote add origin https://gitea.alexandru.macocian.me/Charlie/<name>.git
# 6. Init + first commit
git init -b main
git add .
git status
git diff --cached --stat
git commit -m "Initial import"
git remote add origin https://gitea.alexandru.macocian.me/Charlie/<new>.git
git push -u origin main
# 7. Bring it back up FROM THE NEW LOCATION
sudo docker compose up -d
sudo docker compose ps
sudo docker compose logs -f --tail=50
# 8. Once verified, retire the old dir (do NOT delete data inside it — already moved)
sudo mv "$OLD" "$OLD".retired
# Final removal happens at end of full reorg.
```
## `upstream+overrides`
## `rewrite` — already a git repo
For stacks that are clones of github.com repos we own (e.g. `Gssh`, `MediaServerCompose`, `DirectoryAdmin`).
```bash
cd /mnt/nas/home/<stack>
git remote rename origin upstream
git remote add origin https://gitea.alexandru.macocian.me/Charlie/<name>.git
# create our overlay branch
git checkout -b charlie
# add our files: docker-compose.override.yml, .env.example, custom Dockerfile…
git add docker-compose.override.yml .env.example
git commit -m "Charlie overrides"
git push -u origin charlie
OLD=/mnt/nas/home/<old>
NEW=/mnt/nas/stacks/<new>
DATA=/mnt/nas/data/<new>
sudo mkdir -p "$DATA"
sudo chown -R alex:alex "$DATA"
# 1. Move state out
sudo mv "$OLD"/<state-dir> "$DATA"/ 2>/dev/null || true
# 2. Move the whole git repo to the new location (preserves .git)
sudo mv "$OLD" "$NEW"
sudo chown -R alex:alex "$NEW"
cd "$NEW"
# 3. Repoint origin to Charlie (keep github as a fallback remote)
git remote rename origin github
git remote add origin https://gitea.alexandru.macocian.me/Charlie/<new>.git
# 4. Update compose binds to /mnt/nas/data/<new>/...
$EDITOR docker-compose.yml docker-compose.override.yml
# 5. .gitignore additions for the data dirs we just moved out
$EDITOR .gitignore
# 6. Commit, push to Charlie
git add .
git commit -m "Move bind paths to /mnt/nas/data/<new>"
git push -u origin main
# 7. Bring back up
sudo docker compose up -d
```
The pipeline tracks `origin/charlie`, not upstream's default.
## `mirror` — pull-mirror only, no host action
## `mirror`
In Gitea: New Migration → Pull mirror → upstream URL → name `Charlie/<name>-mirror` → schedule `24h0m0s`.
No host-side action. In Gitea: New Migration → **Pull mirror** → upstream URL → schedule (e.g. daily). Repo lives at `Charlie/<name>` and is read-only.
No filesystem changes on any host.
## Post-migration check
## Post-stack checklist
```bash
git -C /mnt/nas/home/<stack> status # should be clean
git -C /mnt/nas/home/<stack> remote -v # origin → gitea
sudo docker compose ps # still up
```
- [ ] Container is up at the new location.
- [ ] `docker inspect <container> --format '{{ range .Mounts }}{{ .Source }}{{"\n"}}{{ end }}'` confirms binds point at `/mnt/nas/data/<new>/...`.
- [ ] `git -C /mnt/nas/stacks/<new> status` is clean.
- [ ] Old `home/<old>.retired/` exists and is empty of state (only deployment leftovers).
- [ ] Service reachable via Caddy as before.
docs/reorg.md
+78
View File
@@ -0,0 +1,78 @@
# Reorg runbook
The one-time migration from `/mnt/nas/home/` to `/mnt/nas/stacks/` + `/mnt/nas/data/`.
## Phase 0 — preconditions
- [ ] Admin SSH on morgott + melina + mohg.
- [ ] `Charlie/project-charlie` pushed (done).
- [ ] Decided owner host per stack (see [repos.md](repos.md)).
- [ ] Snapshot taken on miquella (Synology snapshot of `volume1/ubuntu`).
## Phase 1 — scaffolding (no downtime)
```bash
# On any one host (NAS is shared).
sudo mkdir -p /mnt/nas/stacks /mnt/nas/data
sudo chown alex:alex /mnt/nas/stacks /mnt/nas/data
```
Optional rename now (still no downtime — these are bulk dirs nothing reads except by full path):
```bash
# Only if you commit to the new names. Update bind paths in compose files BEFORE renaming.
sudo mv /mnt/nas/media-server /mnt/nas/media # if no compose binds it yet at the old path
sudo mv /mnt/nas/ldap /mnt/nas/ldap-certs
```
(Actually: don't rename `media-server` until `mediacompose` is migrated — its compose binds the old path. Leave both renames for the per-stack pass.)
## Phase 2 — create empty Charlie repos in Gitea
Per [repos.md](repos.md), create empty repos:
- All `init` and `rewrite` rows.
- All `mirror` rows (configured as pull-mirrors, see [migration.md](migration.md#mirror)).
No host changes yet.
## Phase 3 — per-stack migration loop
Order: Tier 1 → 5 from [repos.md](repos.md). For each stack on its owner host:
1. `cd /mnt/nas/home/<old> && sudo docker compose down`
2. Run [migration.md](migration.md) recipe (`init` or `rewrite`).
3. Verify checklist at the bottom of [migration.md](migration.md).
4. ✅ on to next.
Pilot: do `ddclient` first end-to-end before touching anything else. If it works, the pattern works.
## Phase 4 — sweep
Once **every** stack is migrated and verified:
```bash
# On miquella or any NFS client:
ls /mnt/nas/home/ # expect: only *.retired/
sudo rm -rf /mnt/nas/home/*.retired
sudo rmdir /mnt/nas/home
# Final layout check
ls /mnt/nas/ # expect: ca, data, ldap-certs, media, stacks, #recycle
```
## Phase 5 — runner stack
Now that everything is git-tracked, design + deploy `Charlie/runners` (separate effort). Pilot redeploy via webhook on `Charlie/ddclient`.
## Rollback strategy
Any stack failing to come up at the new location:
1. `sudo docker compose down` in `/mnt/nas/stacks/<new>/`.
2. `sudo mv /mnt/nas/home/<old>.retired /mnt/nas/home/<old>` (state was moved, not copied — restore is just a path rename of the directory shell).
3. Move state back: `sudo mv /mnt/nas/data/<new>/* /mnt/nas/home/<old>/`.
4. `cd /mnt/nas/home/<old> && sudo docker compose up -d`.
5. Investigate, retry.
Because state was **moved** (not copied), there's only ever one source of truth. No risk of two diverging copies.
docs/repos.md
+40 -39
View File
@@ -1,73 +1,74 @@
# Repo plan — `Charlie/*`
Order of carving. See [inventory.md](inventory.md) for the full list with repo types.
Carving order. See [inventory.md](inventory.md) for the full list. Migration recipe: [migration.md](migration.md). Sequenced runbook: [reorg.md](reorg.md).
## Tier 0 — bootstrap
| Repo | Owner host | Repo type | Why first |
|------|------------|-----------|-----------|
| `project-charlie` | — | new | meta-repo (this) |
| Repo | Owner | Status |
|---|---|---|
| `project-charlie` | — | done |
## Tier 1 — pilot the loop (small, low blast radius)
## Tier 1 — pilot
Goal: prove `git push → webhook → runner → docker compose up -d` end-to-end before touching anything important.
Prove the loop end-to-end. Smallest blast radius.
| Repo | Owner | Source | Repo type |
|------|-------|--------|-----------|
| Repo | Owner | Source | Type |
|---|---|---|---|
| `ddclient` | morgott | `home/ddclient/` | init |
| `homarr` | morgott | `home/homarr/` | init |
| `vaultwarden` | morgott | `home/vaultwarden/` | init |
## Tier 2 — own image, our code
## Tier 2 — our code
| Repo | Owner | Source | Repo type |
|------|-------|--------|-----------|
| Repo | Owner | Source | Type |
|---|---|---|---|
| `gssh` | morgott | `home/Gssh/` | rewrite |
| `directoryadmin` | melina | `home/DirectoryAdmin/` | rewrite |
| `mediacompose` | melina | `home/MediaServerCompose/` | rewrite |
| `server-mgmt-utils` | morgott | `home/ServerManagementUtils/` | rewrite |
| `caddy` | morgott | `home/caddy/` (custom Dockerfile) | init |
## Tier 3 — pinned upstream images, plain init
| Repo | Owner | Source | Repo type |
|------|-------|--------|-----------|
| `ldap` | morgott | `home/open-ldap/` | init |
| `authentik` | morgott | `home/authentik/authentik/` | init |
| `bookstack` | morgott | `home/bookstack/` | init |
| `monica` | morgott | `home/monica/monica/` | init |
| `miniflux` | morgott | `home/miniflux/` | init |
| `planka` | melina | `home/planka/` | init |
| `audiobookshelf` | melina | `home/audiobookshelf/` | init |
| `gitea` | melina | `home/gitea/` | init (deployed by *morgott* runner — never self) |
| Repo | Owner | Source |
|---|---|---|
| `ldap` | morgott | `home/open-ldap/` |
| `authentik` | morgott | `home/authentik/authentik/` |
| `bookstack` | morgott | `home/bookstack/` |
| `monica` | morgott | `home/monica/monica/` |
| `miniflux` | morgott | `home/miniflux/` |
| `planka` | melina | `home/planka/` |
| `audiobookshelf` | melina | `home/audiobookshelf/` |
| `gitea` | melina | `home/gitea/` (deployed by morgott runner — never self) |
## Tier 4 — upstream + overrides
## Tier 4 — was upstream-cloned, now plain init
Two-remote model. See [strategy.md](strategy.md#upstreamoverrides).
These previously lived as upstream-clones with `docker-compose.override.yml`. We're collapsing override → canonical compose. **Hand-merge step required** in [migration.md](migration.md).
| Repo | Owner | Source | Upstream |
|------|-------|--------|----------|
| `invidious` | melina | `home/invidious/` | iv-org/invidious |
| `searxng` | melina | `home/searxng-docker/` | searxng/searxng-docker |
| `victoriametrics` | morgott | `home/VictoriaMetrics/deployment/docker/` | VictoriaMetrics/VictoriaMetrics |
| `mealie` | morgott | `home/mealie/mealie/docker/` | mealie-recipes/mealie |
| `paperless-ngx` | morgott | `home/paperless-ngx/` | paperless-ngx/paperless-ngx |
| Repo | Owner | Source | Image source |
|---|---|---|---|
| `invidious` | melina | `home/invidious/` | `quay.io/invidious/invidious` |
| `searxng` | melina | `home/searxng-docker/` | `searxng/searxng` |
| `victoriametrics` | morgott | `home/VictoriaMetrics/deployment/docker/` | `victoriametrics/victoria-metrics` |
| `mealie` | morgott | `home/mealie/mealie/docker/` | `ghcr.io/mealie-recipes/mealie` |
| `paperless-ngx` | morgott | `home/paperless-ngx/` | `ghcr.io/paperless-ngx/paperless-ngx` |
## Tier 5 — long tail / utilities
| Repo | Owner | Source | Repo type |
|------|-------|--------|-----------|
| Repo | Owner | Source | Type |
|---|---|---|---|
| `otelcollector` | shared | `home/otelcollector/` | init |
| `portainer` | morgott | `home/portainer/` | init |
| `portainer-agent` | shared | `home/portainer-agent/` | init |
| `server-mgmt-utils` | morgott | `home/ServerManagementUtils/` | rewrite |
| `shell` | — | `home/shell/` | init (config-only, not deployed) |
| `watchtower` | morgott | `home/watchtower/` | init then **archive after Diun + pipelines land** |
| `shell` | — (config-only) | `home/shell/` | init |
| `first-time-setup` | — (docs-only) | `home/first-time-setup.md` | init |
| `watchtower` | morgott | `home/watchtower/` | init then archive |
## Mirrors (no overrides, backup only)
## Mirrors
Pull-mirrors under `Charlie/mirrors/` (or as plain `Charlie/<name>-mirror`):
- `nitter`, plus any of the Tier 4 upstreams we want a guaranteed offline copy of.
See [inventory.md](inventory.md#mirrors). Configured in Gitea UI; no host action.
## Deferred / dormant
## Deferred
`plist/` — classify before deciding.
`nitter/`, `vrising/` — on disk, not running. Leave as-is or archive.
docs/roadmap.md
+11 -11
View File
@@ -1,19 +1,19 @@
# Roadmap
## Phase 1 — CI plumbing
## Phase 1 — reorg + CI plumbing
- [x] Inventory `/mnt/nas/` stacks.
- [ ] Push `Charlie/project-charlie` (this repo).
- [ ] Set up Gitea pull-mirrors for relevant upstreams (see [strategy.md](strategy.md#mirror)).
- [ ] Design runner stack (host docker socket; secrets sourcing TBD — candidate: vaultwarden).
- [ ] Stand up runners on morgott + melina, register org-scope to `Charlie`.
- [ ] Pilot end-to-end on `Charlie/ddclient`: push → webhook → redeploy.
- [x] Push `Charlie/project-charlie`.
- [ ] Snapshot `volume1/ubuntu` on miquella.
- [ ] Create empty `Charlie/<stack>` repos per [repos.md](repos.md).
- [ ] Create Gitea pull-mirrors per [inventory.md](inventory.md#mirrors).
- [ ] Walk [reorg.md](reorg.md) Tier 1 (pilot: `ddclient`).
- [ ] Walk Tiers 2–5.
- [ ] Delete `/mnt/nas/home/`.
- [ ] Design + deploy runner stack (`Charlie/runners`).
- [ ] First end-to-end pipeline: push to `Charlie/ddclient` → webhook → redeploy on morgott.
## Phase 2 — Per-app repos
Walk Tier 1 → Tier 5 in [repos.md](repos.md), one stack at a time. Each migration follows [migration.md](migration.md).
## Phase 3 — Automation
## Phase 2 — Automation
- Diun watching registries → webhook → `redeploy` workflow with `force-pull`.
- Retire watchtower.
docs/strategy.md
+20 -17
View File
@@ -8,43 +8,46 @@ Pull-based GitOps via Gitea Actions runners on the owner host.
dev box ── git push ──► Gitea (Charlie/<stack>) ── webhook ──► runner on owner host
│
▼
git -C /mnt/nas/home/<stack> pull --ff-only
git -C /mnt/nas/stacks/<stack> pull --ff-only
docker compose pull && docker compose up -d
```
The working tree on the owner host **is** the deployment. `git status` on the host is the source-of-truth diff against `Charlie/<stack>`.
The git working tree at `/mnt/nas/stacks/<stack>/` **is** the deployment. Runtime state lives separately under `/mnt/nas/data/<stack>/` (see [layout.md](layout.md)).
## Rules
- One owner host per stack. Only that host's runner ever touches the working tree.
- No hand-edits on the host once a stack is repo-managed. `git pull --ff-only` will refuse to clobber.
- `.gitignore` first, `git add` second. Runtime state (`data/`, `db-data/`, `cache/`, `*-data/`) never enters git.
- Runtime state never enters git. Compose files bind into `/mnt/nas/data/<stack>/`, not into the working tree.
- `.env` ignored, `.env.example` tracked. Secrets stay out.
- Image tags pinned. No `:latest`.
- Cross-host self-update: the runner that deploys `Charlie/gitea` lives on morgott; the runner that updates the morgott runner lives on melina. A runner never redeploys itself mid-job.
- Cross-host self-update: the runner that deploys `Charlie/gitea` lives on morgott; the runner that updates morgott's runner lives on melina. A runner never redeploys itself mid-job.
## Repo types
Three. (We dropped `upstream+overrides` — see [Why no upstream cloning](#why-no-upstream-cloning).)
### `init`
Stack is currently just a NAS folder. `git init` in place on the owner host, push to `Charlie/<name>`. Procedure: see [migration.md](migration.md).
Stack is currently just a NAS folder. `git init` in the new `/mnt/nas/stacks/<stack>/`, push to `Charlie/<name>`. Recipe: [migration.md](migration.md#init).
### `rewrite`
Stack is already a clone (typically of a github.com repo we own). `git remote set-url origin https://gitea.alexandru.macocian.me/Charlie/<name>` (and optionally keep github as `github` remote). Push.
### `upstream+overrides`
Stack is a clone of an upstream we don't author (e.g. `invidious`, `searxng-docker`, `VictoriaMetrics`). Two remotes:
- `upstream` — read-only tracking remote.
- `origin` — `Charlie/<name>`, contains *only* our overrides as commits on top: `docker-compose.override.yml`, `.env.example`, custom `Dockerfile`s, optional patches.
Updating the upstream is `git fetch upstream && git rebase upstream/<branch>` (or `merge`), then push to `origin`. Pipeline runs on push to `origin/main`.
Code we author, currently a clone of a github.com repo. Move to `/mnt/nas/stacks/<stack>/`, `git remote set-url origin` to Charlie. Recipe: [migration.md](migration.md#rewrite).
### `mirror`
Pure backup. Configured via Gitea's **pull mirror** feature (Repo → Settings → Mirror). Gitea polls the upstream on a schedule and pushes nothing back. Useful for upstreams we want guaranteed offline access to even when we're not customising them.
Pull-mirror of an upstream we don't customise (or want as offline backup). Configured via Gitea's pull-mirror feature. Read-only, never deployed from. Naming: `Charlie/<name>-mirror`.
## Why no upstream cloning
Previously we considered cloning upstream's source on the host and layering a `docker-compose.override.yml`. Dropped because:
- For every "upstream image" stack (`invidious`, `searxng`, `victoriametrics`, `mealie`, `paperless-ngx`), we only ever ran the published image. Their source tree is dead weight on the host.
- Override-on-template means the upstream's compose can change under us silently.
- Two-remote / rebase / force-push dance was the price for dubious value.
Instead: our `docker-compose.yml` references the published image directly. It is the only compose. Upstream's compose is a *reference document* we read in their mirror repo when we want to see what changed.
## Image-update flow (Phase 3)
@@ -60,4 +63,4 @@ Open question. Candidate: vaultwarden as the source, with a runner-side fetch st
- **Komodo**: nice, also pull-based, also relocates working trees. Reconsider if we ever want a UI.
- **Dockge**: file mgmt only, no git wiring.
The runner-driven pattern keeps the existing `/mnt/nas/home/<stack>/` layout and adds nothing the team doesn't already understand.
The runner-driven pattern keeps us in plain `git` + `docker compose` and adds nothing the team doesn't already understand.