sherlock/plan.md

Charlie Copilot "Sherlock" — research + plan

Problem restatement

Make Copilot CLI a first-class operator on the Charlie homelab:

• Repos on Gitea (gitea.alexandru.macocian.me).
• Hosts on Ubuntu, no direct SSH — only via ~/Dev/gssh (OIDC-authenticated WebSocket SSH proxy, ephemeral certs signed by the in-house CA).
• Identity is Authentik (id.alexandru.macocian.me) — OIDC/OAuth2/SAML.
• Want Copilot to call into Gitea, Grafana, ssh (via gssh), and "anything else" without me pasting tokens.
• Some downstream services use their own OAuth (not federated via Authentik) → need a way to do a one-off browser dance and cache that token too.

The desired "sherlock" sits between Copilot CLI and N service-specific MCPs, handling auth once and injecting per-service credentials.

---

Research findings

1. What Copilot CLI already supports (verified locally)

Looking at ~/.copilot/:

• /mcp slash command manages MCP servers.

• ~/.copilot/mcp-oauth-config/ shows Copilot CLI already implements the full MCP 2025-06-18 authorization spec for HTTP servers: OAuth 2.1 + PKCE, RFC 9728 protected-resource metadata discovery, RFC 7591 dynamic client registration. Sample entry:

  {
    "serverUrl": "https://icm-mcp-prod.azure-api.net/v1/",
    "authorizationServerUrl": "https://login.microsoftonline.com/.../v2.0",
    "clientId": "aebc6443-...",
    "redirectUri": "http://127.0.0.1:34573/",
    ...
  }
  

  → Copilot opens a loopback redirect, runs the browser flow itself, caches tokens.

• STDIO MCP servers — per spec — read credentials from environment variables (no in-band OAuth). This is the natural injection point for the sherlock.

Implication: we don't need to reimplement OAuth inside the MCPs we build. We have two clean options per MCP:

Transport	How auth happens
HTTP MCP	Copilot does OAuth 2.1 against the MCP itself (Authentik = AS, the MCP = resource server).
STDIO MCP	The sherlock spawns it and injects a per-service token via env.

2. MCP authorization spec (2025-06-18) — the relevant rules

• HTTP MCPs MUST implement RFC 9728 Protected Resource Metadata; clients MUST discover the AS via WWW-Authenticate on a 401 or /.well-known/oauth-protected-resource.
• Authorization servers MUST implement OAuth 2.1 + PKCE; SHOULD support RFC 7591 dynamic client registration.
• STDIO transports SHOULD NOT do in-band auth — read from env.
• Tokens MUST be audience-bound (the MCP cannot reuse a Copilot-issued token to call downstream APIs blindly — that's the "confused deputy" the spec calls out).

3. Industry / prior art for "credential brokers for AI agents"

Source	Pattern	Relevance
IETF draft "Credential Broker for Agents" (CB4A)	A broker issues short-lived, narrowly-scoped proxy credentials to agents; agents never hold long-lived OAuth. Policy decision is separated from credential delivery. Builds on SPIFFE concepts.	Exact match for what we want.
TrueFoundry "OAuth 2.0 for MCP" architecture	Splits concerns into (a) inbound auth ("who is calling the MCP?"), (b) access control ("what tools can they use?"), (c) outbound auth ("which downstream OAuth token to inject?"). Per-(user, server) grants in a vault with atomic refresh.	Concrete blueprint for the sherlock daemon.
HashiCorp Vault	Long-standing pattern: brokers short-lived dynamic creds for cloud IAM, DBs, SSH (Vault SSH CA).	Vault SSH CA matches gssh's ephemeral-cert approach. We could lean on Vault or roll an Authentik-native equivalent.
1Password op run / Connect	Wraps a child process and injects secrets as env vars from a vault — the simplest possible "broker".	Mental model for the sherlock copilot wrapper.
sops + age	Already used in project-charlie for secrets at rest.	Good for bootstrapping the broker's own client-credentials, not for live user tokens.
RFC 8693 OAuth Token Exchange	The standardized way to swap one token for another (e.g. Authentik ID token → service-scoped access token).	This is the kernel of the sherlock's "exchange" tool. Authentik supports token exchange via its OAuth2 provider.

4. What we already have in Charlie that maps onto this

• Authentik = the AS for the whole sherlock. Already federates Gitea, Grafana, Caddy-protected apps (per docs/identity.md).
• gssh = a working example of "OIDC → ephemeral cert → proxy to host". This is the model for the SSH MCP: the MCP doesn't hold SSH keys, it asks gssh for a short-lived cert each call.
• sops/age per host = how the broker daemon could bootstrap its own client_secret.
• Caddy + caddy-security = ready-made reverse proxy with OIDC if we want the sherlock to be an HTTP MCP behind sherlock.alexandru.macocian.me.

---

Architectural options

Option A — "Wrapper only" (simplest)

A sherlock shell command that:

1. Ensures the user has a fresh Authentik token (browser flow, cached in ~/.config/sherlock/token).
2. For each configured service, does a token exchange to mint a service-scoped token.
3. Writes a per-session MCP config to $XDG_RUNTIME_DIR/sherlock/mcp.json listing STDIO MCPs with the right env block.
4. execs copilot --mcp-config ….

Pros: ~200 lines of code, no daemon, no new long-running service. Cons: every Copilot launch re-exchanges tokens; no shared cache across terminals; one-off non-Authentik OAuth flows are awkward (no place to host the callback persistently).

Option B — "Sherlock MCP" (single HTTP MCP)

One HTTP MCP server sherlock-mcp, hosted behind Caddy. Copilot does OAuth 2.1 against it natively (using the machinery we already saw in ~/.copilot/mcp-oauth-config/). The sherlock exposes tools like gitea.list_repos, grafana.query, ssh.run(host, cmd), oauth.consent(service).

Pros: single integration point in Copilot; works from any machine; reuses Copilot's OAuth client. Cons: every new tool is a code change inside the sherlock; tool surface gets fat; we lose the modularity of having separate MCPs per service.

Option C — Hybrid: broker daemon + thin stdio MCPs + tiny wrapper (recommended)

┌────────────────────────────────────────────────────────────┐
│  user                                                      │
│    │                                                       │
│    │ `sherlock copilot`                                    │
│    ▼                                                       │
│  ┌─────────────┐  spawns, injects env  ┌──────────────┐    │
│  │ copilot CLI │ ────────────────────▶ │ gitea-mcp    │    │
│  │             │                       │ grafana-mcp  │    │
│  │             │                       │ gssh-mcp     │    │
│  └─────┬───────┘                       └──────┬───────┘    │
│        │ unix socket                          │            │
│        ▼                                      ▼            │
│  ┌──────────────────────────────────────────────────┐      │
│  │ sherlock-broker (systemd --user daemon)          │      │
│  │  - owns the Authentik OAuth state                │      │
│  │  - per-service token cache + refresh             │      │
│  │  - RFC 8693 token exchange                       │      │
│  │  - hosts loopback HTTP for browser callbacks     │      │
│  │  - exposes sherlock-mcp (stdio) for consent flow │      │
│  └────────────────────┬─────────────────────────────┘      │
│                       │ OIDC / token exchange              │
│                       ▼                                    │
│                 Authentik (id.alexandru.macocian.me)       │
└────────────────────────────────────────────────────────────┘

Pieces:

1. sherlock-broker — long-running per-user daemon (systemd --user unit).

   • Listens on a Unix socket ($XDG_RUNTIME_DIR/sherlock.sock).
   • Owns the Authentik OAuth client (PKCE; loopback redirect on a stable port).
   • For each registered service, knows: AS, scopes, exchange mode (federated-via-Authentik | own-oauth | static-pat).
   • Performs RFC 8693 token exchange against Authentik for downstream service tokens.
   • For non-Authentik services, runs the standard auth-code+PKCE dance in a browser and caches the result.
   • Atomic refresh with single-flight to avoid 401 storms.
   • Exposes a tiny RPC: get_token(service, scopes) → { token, expires_at }.

2. Service MCPs — one stdio binary per service (gitea-mcp, grafana-mcp, gssh-mcp, …).

   • Each is thin: it just wraps the service's API and reads XXX_TOKEN from env on startup and refreshes via the broker socket when a 401 comes back.
   • gssh-mcp shells out to ~/Dev/gssh with a short-lived OIDC token from the broker — mirrors how gssh already works.

3. sherlock-mcp — a stdio MCP exposed to Copilot just for the human-in-the-loop stuff:

   • oauth.consent(service) → opens a browser, waits for the broker to receive the callback, returns success/failure.
   • auth.whoami(), auth.list_services(), auth.revoke(service).
   • This is the only MCP that needs UI interaction; the others stay fully non-interactive.

4. sherlock CLI — the wrapper.

   • sherlock login — kick off the Authentik flow up front.
   • sherlock status — show cached tokens / expiry / registered services.
   • sherlock copilot [...] — ensure the broker is up and logged in, render a per-session MCP config, exec copilot "$@".
   • sherlock add-service <name> — register a new service (writes a small TOML/YAML entry that the broker hot-reloads).

Pros: each MCP stays small and replaceable; tokens are cached once and shared across terminals; non-Authentik OAuth flows have a stable home (the broker's loopback listener); matches CB4A / TrueFoundry guidance; mirrors the gssh pattern already proven in the homelab. Cons: more moving parts than A; needs a systemd unit and a stable socket path.

---

Recommendation

Go with Option C, but build it in slices so we have something usable after each slice.

Phased plan

Sliced finer than the original draft so the first end-to-end demo (login + wrapper) lands before any service MCP work.

Phase 0 — bootstrap

• Create Charlie/sherlock repo on Gitea; clone to ~/Dev/charlie/sherlock/.
• Sherlock-flavoured scaffold (NOT the per-app stack scaffold — sherlock is operator-side tooling, no docker-compose.yml, no deploy-stack.yml caller):
  • README.md (TOC only), .gitignore (Go), go.mod (gitea.alexandru.macocian.me/Charlie/sherlock, Go 1.23).
  • Directory skeleton cmd/{sherlock,sherlock-broker}/, internal/{broker,authn,agent,agent/profiles,config,socket,mcp}/.
  • Minimal cmd/sherlock/main.go with version subcommand so CI has something to build.
  • .gitea/workflows/ci.yaml — go vet + go test -race + go build.
• docs/ with one file per topic: architecture.md, auth-model.md, service-registry.md, agent-profiles.md, gssh-integration.md, conventions.md. Service-registry and agent-profile TOML schemas are documented only — loaders land in Phase 1.
• Built-in Copilot agent profile under internal/agent/profiles/copilot.toml (embed-only; not yet read by code).
• Add a row to project-charlie/docs/inventory.md (new "Operator tooling" section) and link from project-charlie/README.md.

Phase 1 — login broker + agent-agnostic wrapper (no MCPs yet)

Smallest slice that proves the auth path and the wrapper indirection end-to-end. No per-service MCP work in this phase.

• sherlock-broker:
  • Authentik PKCE flow with loopback redirect on a stable port.
  • Persisted token cache (mechanism — age-encrypted blob vs OS keyring — decided at the top of this phase).
  • Unix-socket RPC at $XDG_RUNTIME_DIR/sherlock.sock: get_id_token(), whoami(), status().
  • Single-flight refresh.
• sherlock CLI:
  • sherlock login / sherlock logout / sherlock status.
  • sherlock run <agent> [args…] — loads the agent profile, renders the per-session MCP config (empty in Phase 1), execs the agent's binary with the right argv + env.
  • sherlock copilot [args…] — sugar for sherlock run copilot.
• Agent-profile loader in internal/agent/. Built-in copilot.toml is the only profile shipped; user overrides under ~/.config/sherlock/agents.d/ are picked up. A worked Claude Code profile in docs/agent-profiles.md proves the design is extensible.

Exit criterion: sherlock login then sherlock copilot execs copilot with a (possibly empty) MCP config. sherlock status shows a valid JWT. Dropping a claude.toml under agents.d/ makes sherlock claude start working with no rebuild.

Phase 2 — Gitea MCP (first real service)

• gitea-mcp (stdio): list_repos, get_file, list_issues, create_issue, search_code.
• Broker performs RFC 8693 token exchange against Authentik (or passthrough of the ID token if audiences align).
• The wrapper renders an MCP config listing gitea-mcp with the right env.

Exit criterion: sherlock copilot then > list my charlie repos works, no PAT pasted anywhere. Per-user audit trail in Gitea logs.

Phase 3 — gssh MCP

Reuses the existing gssh server (gssh.alexandru.macocian.me) — sherlock does NOT shell out to a local client and does NOT mint SSH certs. Full design in docs/gssh-integration.md.

• gssh-mcp: thin Go HTTP+WebSocket client to gssh's /api/v1/session/* endpoints, authenticated with an Authentik JWT from the broker.
• Tools: ssh.run(host, command, timeout), ssh.list_hosts(), later ssh.put_file(host, path, content).
• Per-host policy is already enforced server-side by gssh's SessionService against JWT claims — sherlock does not duplicate it.
• Completion sentinel strategy: command; echo __SHERLOCK_DONE__$? in the WebSocket shell stream, parsed client-side.

Exit criterion: > on melina, show me docker ps works and the SSH session is logged centrally by gssh.

Phase 4 — Grafana MCP + sherlock-mcp consent server

• grafana-mcp with query, list_dashboards, get_dashboard. Auth via Authentik OIDC (Grafana already federates).
• sherlock-mcp (stdio) with oauth.consent, auth.whoami, auth.list_services, auth.revoke.
• First "non-Authentik" service (something with its own OAuth — e.g. GitHub.com itself, or a SaaS) registered to exercise the consent path end-to-end.

Phase 5 — polish

• systemd --user units for the broker.
• sherlock add-service scaffolder.
• Audit log of every tool call + which token was used (ship to the existing OTEL collector at Charlie/otelcollector).
• Per-service scope minimisation; refuse to issue write-scoped tokens unless the wrapper was launched with --write.
• Docs page docs/sherlock.md in project-charlie, linked from the README.

---

Decided

• Name: sherlock. (Replaces "sherlock" everywhere — sherlock-broker, sherlock-mcp, sherlock copilot …, etc.)
• Repo location: new repo Charlie/sherlock, cloned to ~/Dev/charlie/sherlock/ (sibling of the existing per-app stack repos like gitea/, runners/, etc.). Added to docs/inventory.md under a new "Operator tooling" section and linked from project-charlie/README.md.
• Language: Go. Single static binaries per cmd/<binary>/; best-in-class OAuth/OIDC libs (golang.org/x/oauth2, coreos/go-oidc); official MCP Go SDK; native UDS + systemd integration.
• Phase 2 Gitea auth model: broker mints an OIDC token as the operator via Authentik. Every MCP call shows up under the operator in Gitea's audit log. No dedicated sherlock service account.
• Service-registry format: TOML, under ~/.config/sherlock/services.d/*.toml. Hot-reloaded by the broker.
• SSH backend: reuse the existing gssh server. gssh-mcp is a thin JWT-authenticated HTTP+WS client; no local SSH-cert work in sherlock. See docs/gssh-integration.md.
• Wrapper extensibility: agent-agnostic. Each supported agent CLI (Copilot, Claude Code, Aider, …) is described by a TOML profile under ~/.config/sherlock/agents.d/, with built-in defaults embedded in internal/agent/profiles/. sherlock copilot and sherlock claude are sugar for sherlock run copilot / sherlock run claude. Adding a new agent MUST be a TOML drop-in, never a code change.
• Docs convention: README.md is TOC only; every topic lives as its own file under docs/. New concerns get new files, never appended sections.
• Token persistence (Phase 1): OS keyring via zalando/go-keyring. Strict pre-flight (keyring.Probe()) at startup of both sherlock and sherlock-broker; fail fast with exit code 3 and a per-OS hint if Secret Service / Keychain / Credential Manager is unavailable. See docs/storage.md.
• RPC framing (Phase 1): JSON-over-newline on the UDS at $XDG_RUNTIME_DIR/sherlock.sock. One JSON object per line, both directions. Debuggable with socat/nc. See docs/rpc.md.
• Broker lifecycle (Phase 1): forked child process. The wrapper CLI dials the socket; if no one is listening it fork+execs sherlock-broker daemon with setsid, polls the socket, then proceeds. Per-user PID-file flock prevents double-start; broker auto-exits after SHERLOCK_BROKER_IDLE (default 1h). No systemd in Phase 1.
• Loopback redirect port: 127.0.0.1:6990 (unassigned per IANA). The actual Authentik OAuth2 provider config is deferred — the broker ships with empty defaults and login_start returns not_configured until SHERLOCK_AUTHENTIK_ISSUER / SHERLOCK_AUTHENTIK_CLIENT_ID are set. End-to-end flow is exercised by an integration test against a stub Authentik (internal/authn/flow_test.go).

Still open

Tracked here and revisited at the top of the relevant phase.

1. Phase 2 — Authentik provider creation: stand up the actual sherlock OAuth2 provider in Authentik and document the env vars in docs/storage.md (carry-over from the Phase 1 deferral above).
2. Phase 3 — gssh completion semantics: keep the client-side __SHERLOCK_DONE__ sentinel, or land a server-side one-shot run-and-close mode in gssh.