Charlie/project-charlie
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial scaffold: docs, inventory, strategy, migration recipes

Browse Source
This commit is contained in:
Alexandru Victor Macocian
2026-05-03 13:31:05 +02:00
commit 615575b087
11 changed files with 432 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+15
View File
@@ -0,0 +1,15 @@
# Catch-all for env files. Per-stack .gitignores also list these explicitly
# so they're protected even if a stack is copied out of the repo.
.env
**/.env
**/.env.local
**/.env.*.local
# Runner state
**/data/
# Editor / OS junk
.DS_Store
.idea/
.vscode/
*.swp
README.md
+21
View File
@@ -0,0 +1,21 @@
# Project Charlie
Centralized source of truth for the homelab. Per-app stacks live as repos under the `Charlie` org in Gitea; this repo orchestrates them.
## Docs
- [Topology](docs/topology.md)
- [Hosts](docs/hosts.md)
- [Identity & access](docs/identity.md)
- [Strategy](docs/strategy.md)
- [Conventions](docs/conventions.md)
- [Stack inventory](docs/inventory.md)
- [Repo plan (`Charlie/*`)](docs/repos.md)
- [Migration recipe](docs/migration.md)
- [Roadmap](docs/roadmap.md)
## Assumptions
- `/mnt/nas` from miquella is mounted on every app host (NFS).
- Hosts are bootstrapped per the wiki **First Time Setup** (CA trust, SSSD/LDAP, docker, otel, etc.).
- Long-form prose lives in the wiki (`https://wiki.alexandru.macocian.me`). This repo stays brief.
docs/conventions.md
+10
View File
@@ -0,0 +1,10 @@
# Conventions
- One stack = one repo under `Charlie/`. Repo contains `compose.yaml`, `.env.example`, `.gitea/workflows/deploy.yaml`.
- Secrets never in Git. Use `.env` (host-local, gitignored) or Gitea Actions secrets.
- Persistent state stays under `/mnt/nas/<stack>/` on miquella. Repos define bind paths, never ship data.
- Deploys are pull-based: runner on the target host runs `docker compose pull && up -d`.
- Workflows pin host: `runs-on: [self-hosted, morgott]` or `[self-hosted, melina]`. Never bare `self-hosted`.
- Image tags pinned in compose. No `:latest`.
- A runner never redeploys itself mid-job — cross-host: morgott deploys melina's runner, and vice versa.
- Secrets sourcing: TBD (candidate: vaultwarden).
docs/hosts.md
+14
View File
@@ -0,0 +1,14 @@
# Hosts
| Name | Role | Hardware | NAS |
|------|------|----------|-----|
| Radagon | TP-Link router (WAN) | — | — |
| Marika | OpenWrt router (intranet gateway) | — | — |
| Godfrey | Network switch | — | — |
| Morgott | Caddy + control-plane apps | Intel NUC | yes |
| Melina | Gitea + media stack + apps | Intel NUC, 4c/31G | yes |
| Mohg | apps | Fujitsu Esprimo | yes |
| Malenia | apps | Mac Mini 2012 | — |
| Miquella | NAS, exports `/volume1/ubuntu` → `/mnt/nas` (NFS) | Synology DSM | source |
Bootstrap procedure: wiki → **First Time Setup**.
docs/identity.md
+15
View File
@@ -0,0 +1,15 @@
# Identity & access
| Service | URL | Purpose |
|---------|-----|---------|
| Authentik | https://id.alexandru.macocian.me/ | IdP (OIDC / OAuth2 / SAML) |
| Caddy auth | https://auth.alexandru.macocian.me/ | `caddy-security`, fronts protected apps |
| Web terminal | https://terminal.alexandru.macocian.me/ | Browser SSH via ephemeral signed certs |
## Terminal flow
OAuth → Authentik → backend mints ephemeral SSH cert (signed by CA trusted on every host) → SSH proxied over WebSocket to JS frontend.
CA material lives under `/mnt/nas/ca/` and is synced to hosts by `sync-ca.sh`.
Host logins / sudo are SSSD-backed against `open-ldap`; LDAP groups drive permissions.
docs/inventory.md
+72
View File
@@ -0,0 +1,72 @@
# Stack inventory
Source: `docker compose ls -a` on each host + `find /mnt/nas -name 'docker-compose.y*ml'`. Compose paths under `/mnt/nas/home/` unless noted.
**Repo type:**
- `init` — no git history yet; `git init` in place, push to `Charlie/<name>`.
- `rewrite` — existing git repo (typically on github.com); change `origin` to `Charlie/<name>`.
- `upstream+overrides` — upstream repo we don't author; track `upstream`, add `origin` = `Charlie/<name>` containing only overrides.
- `mirror` — pull-mirror of an upstream repo we use unmodified, kept under `Charlie/<name>` for backup/availability.
## Morgott (owner host)
| Stack | Path | Status | Repo type | Charlie repo |
|-------|------|--------|-----------|--------------|
| caddy | `caddy/` | running | init | `Charlie/caddy` |
| authentik | `authentik/authentik/` | running | init (overrides only) | `Charlie/authentik` |
| open-ldap | `open-ldap/` | running | init | `Charlie/ldap` |
| ddclient | `ddclient/` | running | init | `Charlie/ddclient` |
| vaultwarden | `vaultwarden/` | running | init | `Charlie/vaultwarden` |
| bookstack | `bookstack/` | running | init | `Charlie/bookstack` |
| monica | `monica/monica/` | running | init | `Charlie/monica` |
| miniflux | `miniflux/` | running | init | `Charlie/miniflux` |
| mealie | `mealie/mealie/docker/` | running | upstream+overrides | `Charlie/mealie` |
| homarr | `homarr/` | running | init | `Charlie/homarr` |
| paperless-ngx | `paperless-ngx/` | running | upstream+overrides | `Charlie/paperless-ngx` |
| portainer | `portainer/` | running | init | `Charlie/portainer` |
| portainer-agent | `portainer-agent/` | running | init | `Charlie/portainer-agent` |
| otelcollector | `otelcollector/` | running | init | `Charlie/otelcollector` |
| VictoriaMetrics | `VictoriaMetrics/deployment/docker/` | running (`docker`) | upstream+overrides | `Charlie/victoriametrics` |
| watchtower | `watchtower/` | running | init | `Charlie/watchtower` (retire after pipelines) |
| Gssh | `Gssh/` | running | rewrite | `Charlie/gssh` |
## Melina (owner host)
| Stack | Path | Status | Repo type | Charlie repo |
|-------|------|--------|-----------|--------------|
| gitea | `gitea/` | running | init | `Charlie/gitea` |
| MediaServerCompose | `MediaServerCompose/` | running | rewrite | `Charlie/mediacompose` |
| invidious | `invidious/` | running | upstream+overrides | `Charlie/invidious` |
| searxng-docker | `searxng-docker/` | running | upstream+overrides | `Charlie/searxng` |
| planka | `planka/` | running | init | `Charlie/planka` |
| audiobookshelf | `audiobookshelf/` | running | init | `Charlie/audiobookshelf` |
| DirectoryAdmin | `DirectoryAdmin/` | running | rewrite | `Charlie/directoryadmin` |
| portainer-agent | `portainer-agent/` | running | shared with morgott | `Charlie/portainer-agent` |
| otelcollector | `otelcollector/` | running | shared with morgott | `Charlie/otelcollector` |
## Mohg
TBD — re-run inventory.
## Non-stack dirs (kept on NAS, not stack repos)
| Path | Purpose | Action |
|------|---------|--------|
| `/mnt/nas/ca/` | host CA material (ssh, ldap.lan) | leave; consider `Charlie/ca-tooling` for `sync-ca.sh` only |
| `/mnt/nas/ldap/certs/` | LDAP server certs | leave |
| `/mnt/nas/media-server/` | bulk media (downloads, movies, tv) | leave; bind target |
| `/mnt/nas/portainer/data/` | portainer state | leave |
| `/mnt/nas/home/shell/` | shared shell config (`profile.d`, `inputrc`) | `Charlie/shell` (init) |
| `/mnt/nas/home/plist/` | unclear | classify before action |
| `/mnt/nas/home/ServerManagementUtils/` | filebeat / metricbeat / logstash / orchestrator | rewrite → `Charlie/server-mgmt-utils` |
## Mirror-only (Charlie hosts a backup mirror, no overrides)
Useful for upstreams we may patch later or want offline copies of:
- `invidious`, `searxng-docker`, `VictoriaMetrics`, `mealie`, `paperless-ngx`, `nitter`.
(See [strategy.md](strategy.md) on Gitea pull-mirror config.)
## Removed since last scan
`apache-guacamole`, `docker-dnsmasq`, `Docker-Minecraft-Paper`, `elasticsearch/`, `elk/`, `ferdium-server`, `LibreChat`, `localai`, `nitter` (still on disk, not running), `openobserve`, `reranker`, `scraper`, `step-ca`, `vrising` (game data only), `xstarbound`. `nitter` and `vrising` directories remain on disk but no longer have running stacks.
docs/migration.md
+85
View File
@@ -0,0 +1,85 @@
# Migration — NAS folder → `Charlie/<stack>` repo
For `init`-type stacks. Run on the owner host. Container keeps running throughout.
```bash
# 0. In Gitea UI: create empty Charlie/<name>, private, uninitialized.
cd /mnt/nas/home/<stack>
ls -la && du -sh */ 2>/dev/null # spot the heavy state dirs
git init -b main
# .gitignore BEFORE first add — never stage runtime state
$EDITOR .gitignore # see template below
cp .env .env.example
$EDITOR .env.example # redact every secret
git add .
git status
git diff --cached --stat
# If state dirs slipped through: fix .gitignore, `git rm --cached -r <dir>`, repeat.
git commit -m "Initial import from /mnt/nas/home/<stack>"
git remote add origin https://gitea.alexandru.macocian.me/Charlie/<name>.git
git push -u origin main
```
## `.gitignore` template
```gitignore
# runtime state
data/
db-data/
appdata/
*-data/
cache/
logs/
*.log
*.sqlite*
# secrets
.env
.env.*
!.env.example
```
Tune per stack — e.g. `paperless-ngx` also has `consume/`, `export/`, `media/`, `media-bk/`.
## `rewrite`-type
```bash
cd /mnt/nas/home/<stack>
git remote rename origin github 2>/dev/null || true
git remote add origin https://gitea.alexandru.macocian.me/Charlie/<name>.git
git push -u origin main
```
## `upstream+overrides`
```bash
cd /mnt/nas/home/<stack>
git remote rename origin upstream
git remote add origin https://gitea.alexandru.macocian.me/Charlie/<name>.git
# create our overlay branch
git checkout -b charlie
# add our files: docker-compose.override.yml, .env.example, custom Dockerfile…
git add docker-compose.override.yml .env.example
git commit -m "Charlie overrides"
git push -u origin charlie
```
The pipeline tracks `origin/charlie`, not upstream's default.
## `mirror`
No host-side action. In Gitea: New Migration → **Pull mirror** → upstream URL → schedule (e.g. daily). Repo lives at `Charlie/<name>` and is read-only.
## Post-migration check
```bash
git -C /mnt/nas/home/<stack> status # should be clean
git -C /mnt/nas/home/<stack> remote -v # origin → gitea
sudo docker compose ps # still up
```
docs/repos.md
+73
View File
@@ -0,0 +1,73 @@
# Repo plan — `Charlie/*`
Order of carving. See [inventory.md](inventory.md) for the full list with repo types.
## Tier 0 — bootstrap
| Repo | Owner host | Repo type | Why first |
|------|------------|-----------|-----------|
| `project-charlie` | — | new | meta-repo (this) |
## Tier 1 — pilot the loop (small, low blast radius)
Goal: prove `git push → webhook → runner → docker compose up -d` end-to-end before touching anything important.
| Repo | Owner | Source | Repo type |
|------|-------|--------|-----------|
| `ddclient` | morgott | `home/ddclient/` | init |
| `homarr` | morgott | `home/homarr/` | init |
| `vaultwarden` | morgott | `home/vaultwarden/` | init |
## Tier 2 — own image, our code
| Repo | Owner | Source | Repo type |
|------|-------|--------|-----------|
| `gssh` | morgott | `home/Gssh/` | rewrite |
| `directoryadmin` | melina | `home/DirectoryAdmin/` | rewrite |
| `mediacompose` | melina | `home/MediaServerCompose/` | rewrite |
| `caddy` | morgott | `home/caddy/` (custom Dockerfile) | init |
## Tier 3 — pinned upstream images, plain init
| Repo | Owner | Source | Repo type |
|------|-------|--------|-----------|
| `ldap` | morgott | `home/open-ldap/` | init |
| `authentik` | morgott | `home/authentik/authentik/` | init |
| `bookstack` | morgott | `home/bookstack/` | init |
| `monica` | morgott | `home/monica/monica/` | init |
| `miniflux` | morgott | `home/miniflux/` | init |
| `planka` | melina | `home/planka/` | init |
| `audiobookshelf` | melina | `home/audiobookshelf/` | init |
| `gitea` | melina | `home/gitea/` | init (deployed by *morgott* runner — never self) |
## Tier 4 — upstream + overrides
Two-remote model. See [strategy.md](strategy.md#upstreamoverrides).
| Repo | Owner | Source | Upstream |
|------|-------|--------|----------|
| `invidious` | melina | `home/invidious/` | iv-org/invidious |
| `searxng` | melina | `home/searxng-docker/` | searxng/searxng-docker |
| `victoriametrics` | morgott | `home/VictoriaMetrics/deployment/docker/` | VictoriaMetrics/VictoriaMetrics |
| `mealie` | morgott | `home/mealie/mealie/docker/` | mealie-recipes/mealie |
| `paperless-ngx` | morgott | `home/paperless-ngx/` | paperless-ngx/paperless-ngx |
## Tier 5 — long tail / utilities
| Repo | Owner | Source | Repo type |
|------|-------|--------|-----------|
| `otelcollector` | shared | `home/otelcollector/` | init |
| `portainer` | morgott | `home/portainer/` | init |
| `portainer-agent` | shared | `home/portainer-agent/` | init |
| `server-mgmt-utils` | morgott | `home/ServerManagementUtils/` | rewrite |
| `shell` | — | `home/shell/` | init (config-only, not deployed) |
| `watchtower` | morgott | `home/watchtower/` | init then **archive after Diun + pipelines land** |
## Mirrors (no overrides, backup only)
Pull-mirrors under `Charlie/mirrors/` (or as plain `Charlie/<name>-mirror`):
- `nitter`, plus any of the Tier 4 upstreams we want a guaranteed offline copy of.
## Deferred / dormant
`plist/` — classify before deciding.
docs/roadmap.md
+22
View File
@@ -0,0 +1,22 @@
# Roadmap
## Phase 1 — CI plumbing
- [x] Inventory `/mnt/nas/` stacks.
- [ ] Push `Charlie/project-charlie` (this repo).
- [ ] Set up Gitea pull-mirrors for relevant upstreams (see [strategy.md](strategy.md#mirror)).
- [ ] Design runner stack (host docker socket; secrets sourcing TBD — candidate: vaultwarden).
- [ ] Stand up runners on morgott + melina, register org-scope to `Charlie`.
- [ ] Pilot end-to-end on `Charlie/ddclient`: push → webhook → redeploy.
## Phase 2 — Per-app repos
Walk Tier 1 → Tier 5 in [repos.md](repos.md), one stack at a time. Each migration follows [migration.md](migration.md).
## Phase 3 — Automation
- Diun watching registries → webhook → `redeploy` workflow with `force-pull`.
- Retire watchtower.
- Drift detection: running compose vs repo compose.
- Backup verification against miquella.
- Cert/secret expiry checks (authentik, caddy, signing CA).
docs/strategy.md
+63
View File
@@ -0,0 +1,63 @@
# Strategy
## Pattern
Pull-based GitOps via Gitea Actions runners on the owner host.
```
dev box ── git push ──► Gitea (Charlie/<stack>) ── webhook ──► runner on owner host
│
▼
git -C /mnt/nas/home/<stack> pull --ff-only
docker compose pull && docker compose up -d
```
The working tree on the owner host **is** the deployment. `git status` on the host is the source-of-truth diff against `Charlie/<stack>`.
## Rules
- One owner host per stack. Only that host's runner ever touches the working tree.
- No hand-edits on the host once a stack is repo-managed. `git pull --ff-only` will refuse to clobber.
- `.gitignore` first, `git add` second. Runtime state (`data/`, `db-data/`, `cache/`, `*-data/`) never enters git.
- `.env` ignored, `.env.example` tracked. Secrets stay out.
- Image tags pinned. No `:latest`.
- Cross-host self-update: the runner that deploys `Charlie/gitea` lives on morgott; the runner that updates the morgott runner lives on melina. A runner never redeploys itself mid-job.
## Repo types
### `init`
Stack is currently just a NAS folder. `git init` in place on the owner host, push to `Charlie/<name>`. Procedure: see [migration.md](migration.md).
### `rewrite`
Stack is already a clone (typically of a github.com repo we own). `git remote set-url origin https://gitea.alexandru.macocian.me/Charlie/<name>` (and optionally keep github as `github` remote). Push.
### `upstream+overrides`
Stack is a clone of an upstream we don't author (e.g. `invidious`, `searxng-docker`, `VictoriaMetrics`). Two remotes:
- `upstream` — read-only tracking remote.
- `origin` — `Charlie/<name>`, contains *only* our overrides as commits on top: `docker-compose.override.yml`, `.env.example`, custom `Dockerfile`s, optional patches.
Updating the upstream is `git fetch upstream && git rebase upstream/<branch>` (or `merge`), then push to `origin`. Pipeline runs on push to `origin/main`.
### `mirror`
Pure backup. Configured via Gitea's **pull mirror** feature (Repo → Settings → Mirror). Gitea polls the upstream on a schedule and pushes nothing back. Useful for upstreams we want guaranteed offline access to even when we're not customising them.
## Image-update flow (Phase 3)
Diun watches registries; on a tag/digest change it POSTs to a Gitea webhook that triggers a `redeploy` workflow with `force-pull: true` on the owner host. Replaces watchtower.
## Secrets
Open question. Candidate: vaultwarden as the source, with a runner-side fetch step that materialises `.env` per job. Decided when we design the runner stack.
## Why not Portainer/Komodo/Dockge
- **Portainer Stacks-from-Git**: works, but no submodule support, opinionated about working-tree location, adds a control plane on top of git.
- **Komodo**: nice, also pull-based, also relocates working trees. Reconsider if we ever want a UI.
- **Dockge**: file mgmt only, no git wiring.
The runner-driven pattern keeps the existing `/mnt/nas/home/<stack>/` layout and adds nothing the team doesn't already understand.
docs/topology.md
+42
View File
@@ -0,0 +1,42 @@
# Topology
```mermaid
graph TD
Internet((Internet))
Radagon{Radagon}
Devices{{Local Devices}}
Marika{Marika}
Godfrey[[Godfrey]]
Morgott[Morgott]
Melina[Melina]
Mohg[Mohg]
Malenia[Malenia]
Miquella[Miquella]
Internet <--> Radagon
Radagon <-->|80/443| Marika
subgraph Extranet
Radagon <--> Devices
end
subgraph Intranet
Marika <--> Godfrey
Godfrey <-->|80/443| Morgott
Melina --> Godfrey
Mohg --> Godfrey
Malenia --> Godfrey
Miquella --> Godfrey
Morgott --> Melina
Morgott --> Mohg
Morgott --> Malenia
Morgott --> Miquella
Melina --> Miquella
Malenia --> Miquella
Mohg --> Miquella
end
```
## Ingress path
`Internet → Radagon → Marika → Morgott:caddy → container (local or sibling host over Godfrey)`.
All `*.alexandru.macocian.me` traffic terminates at Caddy on Morgott.