sherlock/plan.md

Charlie Copilot "Sherlock" — research + plan

Problem restatement

Make Copilot CLI a first-class operator on the Charlie homelab:

• Repos on Gitea (gitea.alexandru.macocian.me).
• Hosts on Ubuntu, no direct SSH — only via ~/Dev/gssh (OIDC-authenticated WebSocket SSH proxy, ephemeral certs signed by the in-house CA).
• Identity is Authentik (id.alexandru.macocian.me) — OIDC/OAuth2/SAML.
• Want Copilot to call into Gitea, Grafana, ssh (via gssh), and "anything else" without me pasting tokens.
• Some downstream services use their own OAuth (not federated via Authentik) → need a way to do a one-off browser dance and cache that token too.

The desired "sherlock" sits between Copilot CLI and N service-specific MCPs, handling auth once and injecting per-service credentials.

---

Research findings

1. What Copilot CLI already supports (verified locally)

Looking at ~/.copilot/:

• /mcp slash command manages MCP servers.

• ~/.copilot/mcp-oauth-config/ shows Copilot CLI already implements the full MCP 2025-06-18 authorization spec for HTTP servers: OAuth 2.1 + PKCE, RFC 9728 protected-resource metadata discovery, RFC 7591 dynamic client registration. Sample entry:

  {
    "serverUrl": "https://icm-mcp-prod.azure-api.net/v1/",
    "authorizationServerUrl": "https://login.microsoftonline.com/.../v2.0",
    "clientId": "aebc6443-...",
    "redirectUri": "http://127.0.0.1:34573/",
    ...
  }
  

  → Copilot opens a loopback redirect, runs the browser flow itself, caches tokens.

• STDIO MCP servers — per spec — read credentials from environment variables (no in-band OAuth). This is the natural injection point for the sherlock.

Implication: we don't need to reimplement OAuth inside the MCPs we build. We have two clean options per MCP:

Transport	How auth happens
HTTP MCP	Copilot does OAuth 2.1 against the MCP itself (Authentik = AS, the MCP = resource server).
STDIO MCP	The sherlock spawns it and injects a per-service token via env.

2. MCP authorization spec (2025-06-18) — the relevant rules

• HTTP MCPs MUST implement RFC 9728 Protected Resource Metadata; clients MUST discover the AS via WWW-Authenticate on a 401 or /.well-known/oauth-protected-resource.
• Authorization servers MUST implement OAuth 2.1 + PKCE; SHOULD support RFC 7591 dynamic client registration.
• STDIO transports SHOULD NOT do in-band auth — read from env.
• Tokens MUST be audience-bound (the MCP cannot reuse a Copilot-issued token to call downstream APIs blindly — that's the "confused deputy" the spec calls out).

3. Industry / prior art for "credential brokers for AI agents"

Source	Pattern	Relevance
IETF draft "Credential Broker for Agents" (CB4A)	A broker issues short-lived, narrowly-scoped proxy credentials to agents; agents never hold long-lived OAuth. Policy decision is separated from credential delivery. Builds on SPIFFE concepts.	Exact match for what we want.
TrueFoundry "OAuth 2.0 for MCP" architecture	Splits concerns into (a) inbound auth ("who is calling the MCP?"), (b) access control ("what tools can they use?"), (c) outbound auth ("which downstream OAuth token to inject?"). Per-(user, server) grants in a vault with atomic refresh.	Concrete blueprint for the sherlock daemon.
HashiCorp Vault	Long-standing pattern: brokers short-lived dynamic creds for cloud IAM, DBs, SSH (Vault SSH CA).	Vault SSH CA matches gssh's ephemeral-cert approach. We could lean on Vault or roll an Authentik-native equivalent.
1Password op run / Connect	Wraps a child process and injects secrets as env vars from a vault — the simplest possible "broker".	Mental model for the sherlock copilot wrapper.
sops + age	Already used in project-charlie for secrets at rest.	Good for bootstrapping the broker's own client-credentials, not for live user tokens.
RFC 8693 OAuth Token Exchange	The standardized way to swap one token for another (e.g. Authentik ID token → service-scoped access token).	This is the kernel of the sherlock's "exchange" tool. Authentik supports token exchange via its OAuth2 provider.

4. What we already have in Charlie that maps onto this

• Authentik = the AS for the whole sherlock. Already federates Gitea, Grafana, Caddy-protected apps (per docs/identity.md).
• gssh = a working example of "OIDC → ephemeral cert → proxy to host". This is the model for the SSH MCP: the MCP doesn't hold SSH keys, it asks gssh for a short-lived cert each call.
• sops/age per host = how the broker daemon could bootstrap its own client_secret.
• Caddy + caddy-security = ready-made reverse proxy with OIDC if we want the sherlock to be an HTTP MCP behind sherlock.alexandru.macocian.me.

---

Architectural options

Option A — "Wrapper only" (simplest)

A sherlock shell command that:

1. Ensures the user has a fresh Authentik token (browser flow, cached in ~/.config/sherlock/token).
2. For each configured service, does a token exchange to mint a service-scoped token.
3. Writes a per-session MCP config to $XDG_RUNTIME_DIR/sherlock/mcp.json listing STDIO MCPs with the right env block.
4. execs copilot --mcp-config ….

Pros: ~200 lines of code, no daemon, no new long-running service. Cons: every Copilot launch re-exchanges tokens; no shared cache across terminals; one-off non-Authentik OAuth flows are awkward (no place to host the callback persistently).

Option B — "Sherlock MCP" (single HTTP MCP)

One HTTP MCP server sherlock-mcp, hosted behind Caddy. Copilot does OAuth 2.1 against it natively (using the machinery we already saw in ~/.copilot/mcp-oauth-config/). The sherlock exposes tools like gitea.list_repos, grafana.query, ssh.run(host, cmd), oauth.consent(service).

Pros: single integration point in Copilot; works from any machine; reuses Copilot's OAuth client. Cons: every new tool is a code change inside the sherlock; tool surface gets fat; we lose the modularity of having separate MCPs per service.

Option C — Hybrid: broker daemon + thin stdio MCPs + tiny wrapper (recommended)

┌────────────────────────────────────────────────────────────┐
│  user                                                      │
│    │                                                       │
│    │ `sherlock copilot`                                    │
│    ▼                                                       │
│  ┌─────────────┐  spawns, injects env  ┌──────────────┐    │
│  │ copilot CLI │ ────────────────────▶ │ gitea-mcp    │    │
│  │             │                       │ grafana-mcp  │    │
│  │             │                       │ gssh-mcp     │    │
│  └─────┬───────┘                       └──────┬───────┘    │
│        │ unix socket                          │            │
│        ▼                                      ▼            │
│  ┌──────────────────────────────────────────────────┐      │
│  │ sherlock-broker (systemd --user daemon)          │      │
│  │  - owns the Authentik OAuth state                │      │
│  │  - per-service token cache + refresh             │      │
│  │  - RFC 8693 token exchange                       │      │
│  │  - hosts loopback HTTP for browser callbacks     │      │
│  │  - exposes sherlock-mcp (stdio) for consent flow │      │
│  └────────────────────┬─────────────────────────────┘      │
│                       │ OIDC / token exchange              │
│                       ▼                                    │
│                 Authentik (id.alexandru.macocian.me)       │
└────────────────────────────────────────────────────────────┘

Pieces:

1. sherlock-broker — long-running per-user daemon (systemd --user unit).

   • Listens on a Unix socket ($XDG_RUNTIME_DIR/sherlock.sock).
   • Owns the Authentik OAuth client (PKCE; loopback redirect on a stable port).
   • For each registered service, knows: AS, scopes, exchange mode (federated-via-Authentik | own-oauth | static-pat).
   • Performs RFC 8693 token exchange against Authentik for downstream service tokens.
   • For non-Authentik services, runs the standard auth-code+PKCE dance in a browser and caches the result.
   • Atomic refresh with single-flight to avoid 401 storms.
   • Exposes a tiny RPC: get_token(service, scopes) → { token, expires_at }.

2. Service MCPs — one stdio binary per service (gitea-mcp, grafana-mcp, gssh-mcp, …).

   • Each is thin: it just wraps the service's API and reads XXX_TOKEN from env on startup and refreshes via the broker socket when a 401 comes back.
   • gssh-mcp shells out to ~/Dev/gssh with a short-lived OIDC token from the broker — mirrors how gssh already works.

3. sherlock-mcp — a stdio MCP exposed to Copilot just for the human-in-the-loop stuff:

   • oauth.consent(service) → opens a browser, waits for the broker to receive the callback, returns success/failure.
   • auth.whoami(), auth.list_services(), auth.revoke(service).
   • This is the only MCP that needs UI interaction; the others stay fully non-interactive.

4. sherlock CLI — the wrapper.

   • sherlock login — kick off the Authentik flow up front.
   • sherlock status — show cached tokens / expiry / registered services.
   • sherlock copilot [...] — ensure the broker is up and logged in, render a per-session MCP config, exec copilot "$@".
   • sherlock add-service <name> — register a new service (writes a small TOML/YAML entry that the broker hot-reloads).

Pros: each MCP stays small and replaceable; tokens are cached once and shared across terminals; non-Authentik OAuth flows have a stable home (the broker's loopback listener); matches CB4A / TrueFoundry guidance; mirrors the gssh pattern already proven in the homelab. Cons: more moving parts than A; needs a systemd unit and a stable socket path.

---

Recommendation

Go with Option C, but build it in slices so we have something usable after each slice.

Phased plan

Sliced finer than the original draft so the first end-to-end demo (login + wrapper) lands before any service MCP work.

Phase 0 — bootstrap

• Create Charlie/sherlock repo on Gitea; clone to ~/Dev/charlie/sherlock/.
• Sherlock-flavoured scaffold (NOT the per-app stack scaffold — sherlock is operator-side tooling, no docker-compose.yml, no deploy-stack.yml caller):
  • README.md (TOC only), .gitignore (Go), go.mod (gitea.alexandru.macocian.me/Charlie/sherlock, Go 1.23).
  • Directory skeleton cmd/{sherlock,sherlock-broker}/, internal/{broker,authn,agent,agent/profiles,config,socket,mcp}/.
  • Minimal cmd/sherlock/main.go with version subcommand so CI has something to build.
  • .gitea/workflows/ci.yaml — go vet + go test -race + go build.
• docs/ with one file per topic: architecture.md, auth-model.md, service-registry.md, agent-profiles.md, gssh-integration.md, conventions.md. Service-registry and agent-profile TOML schemas are documented only — loaders land in Phase 1.
• Built-in Copilot agent profile under internal/agent/profiles/copilot.toml (embed-only; not yet read by code).
• Add a row to project-charlie/docs/inventory.md (new "Operator tooling" section) and link from project-charlie/README.md.

Phase 1 — login broker + agent-agnostic wrapper (no MCPs yet)

Smallest slice that proves the auth path and the wrapper indirection end-to-end. No per-service MCP work in this phase.

• sherlock-broker:
  • Authentik PKCE flow with loopback redirect on a stable port.
  • Persisted token cache (mechanism — age-encrypted blob vs OS keyring — decided at the top of this phase).
  • Unix-socket RPC at $XDG_RUNTIME_DIR/sherlock.sock: get_id_token(), whoami(), status().
  • Single-flight refresh.
• sherlock CLI:
  • sherlock login / sherlock logout / sherlock status.
  • sherlock run <agent> [args…] — looks up the agent in the in-binary registry, renders the per-session MCP config (empty in Phase 1), execs the agent's binary with the right argv + env.
  • sherlock copilot [args…] — sugar for sherlock run copilot.
• One Go file per supported agent under internal/agent/, each registering itself via init(). Ships with copilot.go (--additional-mcp-config @<path>, VS Code MCP shape) and claude.go (--mcp-config <path>, .mcp.json shape; strips ANTHROPIC_API_KEY from the child env). See docs/agents.md.

Exit criterion: sherlock login then sherlock copilot execs copilot with a (possibly empty) MCP config. sherlock status shows a valid JWT. Adding a new agent is a new Go file in internal/agent/, ~30 LoC, no schema work.

Phase 2 — Gitea MCP (first real service)

• gitea-mcp (stdio): list_repos, get_file, list_issues, create_issue, search_code.
• Broker performs RFC 8693 token exchange against Authentik (or passthrough of the ID token if audiences align).
• The wrapper renders an MCP config listing gitea-mcp with the right env.

Exit criterion: sherlock copilot then > list my charlie repos works, no PAT pasted anywhere. Per-user audit trail in Gitea logs.

Phase 3 — gssh MCP

Reuses the existing gssh server (gssh.alexandru.macocian.me) — sherlock does NOT shell out to a local client and does NOT mint SSH certs. Full design in docs/gssh-integration.md.

• gssh-mcp: thin Go HTTP+WebSocket client to gssh's /api/v1/session/* endpoints, authenticated with an Authentik JWT from the broker.
• Tools: ssh.run(host, command, timeout), ssh.list_hosts(), later ssh.put_file(host, path, content).
• Per-host policy is already enforced server-side by gssh's SessionService against JWT claims — sherlock does not duplicate it.
• Completion sentinel strategy: command; echo __SHERLOCK_DONE__$? in the WebSocket shell stream, parsed client-side.

Exit criterion: > on melina, show me docker ps works and the SSH session is logged centrally by gssh.

Phase 4 — Grafana MCP + sherlock-mcp consent server

• grafana-mcp with query, list_dashboards, get_dashboard. Auth via Authentik OIDC (Grafana already federates).
• sherlock-mcp (stdio) with oauth.consent, auth.whoami, auth.list_services, auth.revoke.
• First "non-Authentik" service (something with its own OAuth — e.g. GitHub.com itself, or a SaaS) registered to exercise the consent path end-to-end.

Phase 5 — polish

• systemd --user units for the broker.
• sherlock add-service scaffolder.
• Audit log of every tool call + which token was used (ship to the existing OTEL collector at Charlie/otelcollector).
• Per-service scope minimisation; refuse to issue write-scoped tokens unless the wrapper was launched with --write.
• Docs page docs/sherlock.md in project-charlie, linked from the README.

---

Decided

• Name: sherlock. (Replaces "sherlock" everywhere — sherlock-broker, sherlock-mcp, sherlock copilot …, etc.)
• Repo location: new repo Charlie/sherlock, cloned to ~/Dev/charlie/sherlock/ (sibling of the existing per-app stack repos like gitea/, runners/, etc.). Added to docs/inventory.md under a new "Operator tooling" section and linked from project-charlie/README.md.
• Language: Go. Single static binary cmd/sherlock/; best-in-class OAuth/OIDC libs (golang.org/x/oauth2, coreos/go-oidc); official MCP Go SDK; native Unix primitives (flock, syscall.Exec).
• Phase 2 Gitea auth model: the gitea-mcp uses an OIDC token as the operator via Authentik. Every MCP call shows up under the operator in Gitea's audit log. No dedicated sherlock service account.
• Service-registry format: TOML, under ~/.config/sherlock/services.d/*.toml. Loaded at the start of each CLI invocation (no daemon to hot-reload).
• SSH backend: reuse the existing gssh server. gssh-mcp is a thin JWT-authenticated HTTP+WS client; no local SSH-cert work in sherlock. See docs/gssh-integration.md.
• Wrapper extensibility: agent-agnostic. Each supported agent CLI (Copilot, Claude Code, Aider, …) is a single Go file under internal/agent/ that registers itself with the package via init(). sherlock copilot and sherlock claude are sugar for sherlock run copilot / sherlock run claude. Adding a new agent is a small code change, by design — per-CLI quirks (auth subcommands, MCP config schema, env-var quirks) need a real code home. The TOML-overlay design from the Phase 0 plan was tried in Phase 1 and dropped. See docs/agents.md.
• Docs convention: README.md is TOC only; every topic lives as its own file under docs/. New concerns get new files, never appended sections.
• Token persistence: OS keyring via zalando/go-keyring. The only public constructor is keyring.Open() — it probes the keyring (write+read+delete of a sentinel) and returns the live Store or an *UnavailableError whose Hint field carries platform-specific remediation. Fails fast with exit code 3 if Secret Service / Keychain / Credential Manager is unavailable. There is no probe-less escape hatch — the API itself enforces the fail-fast invariant. See docs/storage.md.
• Loopback redirect port: 127.0.0.1:6990 (unassigned per IANA). Bound only for the duration of sherlock login. The actual Authentik OAuth2 provider config is deferred — sherlock ships with empty defaults and sherlock login returns not_configured until SHERLOCK_AUTHENTIK_ISSUER / SHERLOCK_AUTHENTIK_CLIENT_ID are set. End-to-end flow is exercised by an integration test against a stub Authentik (internal/authn/login_test.go).
• No daemon (reverses Phase 1 #9 + #10): the Phase 1 design had a sherlock-broker daemon with UDS RPC (docs/rpc.md), JSON-over-newline framing, fork+setsid lifecycle, PID-file flock, and idle-timeout watcher. It was scrapped in the post-Phase-1 refactor — see docs/architecture.md#why-there-is-no-daemon. Multi-process refresh races are serialised via an exclusive flock() on $XDG_RUNTIME_DIR/sherlock.refresh.lock in internal/authn/refresh.go. Net: -800 LoC, no internal/{broker,socket,rpc}/ packages, no cmd/sherlock-broker/ binary.

Still open

Tracked here and revisited at the top of the relevant phase.

1. Phase 2 — Authentik provider creation: stand up the actual sherlock OAuth2 provider in Authentik and document the env vars in docs/storage.md (carry-over from the Phase 1 deferral above).
2. Phase 2 — own-oauth UX: confirm sherlock auth <service> is the right surface for non-Authentik services like GitHub.com (vs. an interactive sherlock-mcp.oauth.consent flow). The no-daemon refactor pushed this off — without a long-lived listener, an explicit command is the natural way to bind the loopback port.
3. Phase 3 — gssh completion semantics: keep the client-side __SHERLOCK_DONE__ sentinel, or land a server-side one-shot run-and-close mode in gssh.