@@ -1,6 +1,8 @@
|
||||
# sherlock
|
||||
|
||||
Per-operator credential wallet + agent-CLI wrapper for the Charlie homelab. Holds one OAuth session per service in the OS keyring, exposes a shared auth library that MCPs call lazily at startup, and execs the agent CLI of your choice (Copilot today, Claude Code or any other MCP-aware agent tomorrow).
|
||||
Per-operator credential wallet and agent wrapper for the Charlie homelab.
|
||||
Sherlock stores service OAuth sessions in the OS keyring, launches supported
|
||||
agent CLIs with generated MCP config, and lets MCPs authenticate lazily.
|
||||
|
||||
## Docs
|
||||
|
||||
|
||||
+19
-88
@@ -1,97 +1,28 @@
|
||||
# Agents
|
||||
|
||||
How sherlock decides which CLI to spawn when you type
|
||||
`sherlock copilot` or `sherlock claude`, and how to add a new one.
|
||||
Sherlock dispatches built-in agent profiles from `internal/agent/`.
|
||||
|
||||
## Supported today
|
||||
| Agent | CLI | MCP config |
|
||||
| --------- | ------------------ | --------------------------------- |
|
||||
| `copilot` | GitHub Copilot CLI | `--additional-mcp-config @<path>` |
|
||||
| `claude` | Claude Code CLI | `--mcp-config <path>` |
|
||||
|
||||
| Name | Binary | MCP config flag | Notes |
|
||||
|---|---|---|---|
|
||||
| `copilot` | `copilot` (npm `@github/copilot`) | `--additional-mcp-config @<path>` | Augments user's `~/.copilot/mcp-config.json`. JSON shape is the canonical `.mcp.json` schema (`{"mcpServers": ...}`). |
|
||||
| `claude` | `claude` (npm `@anthropic-ai/claude-code`) | `--mcp-config <path>` | Same `{"mcpServers": ...}` shape. `ANTHROPIC_API_KEY` is stripped from the child env so a personal key can't override the sherlock-managed session. |
|
||||
`sherlock <agent> [args...]` and `sherlock run <agent> [args...]` are
|
||||
equivalent. Unknown agent names exit with usage errors.
|
||||
|
||||
## Routing
|
||||
## Spawn behavior
|
||||
|
||||
```
|
||||
sherlock copilot [args...] ⇢ runs copilot
|
||||
sherlock claude [args...] ⇢ runs claude
|
||||
sherlock run copilot [args...] ⇢ same, explicit form
|
||||
sherlock run <unknown> ... ⇢ exit 2 with "unknown agent"
|
||||
sherlock <unknown> ⇢ exit 2 with "unknown subcommand"
|
||||
```
|
||||
On spawn, sherlock resolves installed MCP binaries (`gitea-mcp`, `grafana-mcp`,
|
||||
`gssh-mcp`), skips missing ones with a warning, renders a 0600 `.mcp.json` file
|
||||
under `$XDG_RUNTIME_DIR/sherlock/`, and `exec`s the target agent.
|
||||
|
||||
The `run` form exists for parity with `cargo run` / `npm run`; the
|
||||
bare alias is the daily-driver form.
|
||||
The child mostly inherits the parent environment. The Claude profile strips
|
||||
`ANTHROPIC_API_KEY` so a personal key does not override sherlock-managed
|
||||
behavior.
|
||||
|
||||
## What sherlock does per spawn
|
||||
## Changing agents
|
||||
|
||||
1. `keyring.Open()` — fail fast if the OS keyring isn't available (returns `*UnavailableError` with a remediation `Hint` field).
|
||||
2. Resolve the agent binary on `$PATH`. Friendly error if missing.
|
||||
3. Render the per-agent MCP config to `$XDG_RUNTIME_DIR/sherlock/<agent>.mcp.json` (0600). In Phase 1 the servers map is always empty; Phase 2 populates it from `services.d/`.
|
||||
4. Build the child argv with the agent-specific flag.
|
||||
5. Build the child env: parent env minus per-agent forbids. MCPs spawned by the agent will reach into the OS keyring (via `internal/authn.Ensure`) on their own at startup — sherlock does not pre-authenticate anything.
|
||||
6. `syscall.Exec` — sherlock disappears, the agent takes its place.
|
||||
|
||||
## Adding a new agent
|
||||
|
||||
It's a code change, deliberately. The TOML-overlay design was tried
|
||||
and scrapped: each CLI has enough idiosyncrasies (auth subcommands,
|
||||
permission flags, MCP config schema, env var quirks) that a Go file
|
||||
per agent is honest about the surface area and gives those quirks a
|
||||
real place to live.
|
||||
|
||||
Drop a new file in `internal/agent/`:
|
||||
|
||||
```go
|
||||
// internal/agent/aider.go
|
||||
package agent
|
||||
|
||||
import "gitea.alexandru.macocian.me/amacocian/sherlock/internal/mcp"
|
||||
|
||||
func init() { Register(&aider{}) }
|
||||
|
||||
type aider struct{}
|
||||
|
||||
func (aider) Name() string { return "aider" }
|
||||
func (aider) Description() string { return "Aider AI pair programmer" }
|
||||
|
||||
func (a aider) Spawn(ctx Context, args []string) error {
|
||||
bin, err := LookPath("aider")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// Aider's MCP schema and flag would go here.
|
||||
_ = bin
|
||||
_ = ctx
|
||||
_ = args
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
That's the whole API: `Name`, `Description`, `Spawn`. The CLI picks
|
||||
it up automatically through the `init()` registry call; `sherlock
|
||||
status` shows it; `sherlock aider ...` dispatches.
|
||||
|
||||
## Reusable helpers
|
||||
|
||||
Available to every agent implementation in this package:
|
||||
|
||||
| Helper | Purpose |
|
||||
|---|---|
|
||||
| `LookPath(name)` | `exec.LookPath` with a sherlock-friendly error message. |
|
||||
| `BuildEnv(forbid, set)` | parent env minus `forbid`, plus `set`. |
|
||||
| `DefaultExecer` | the package-level `Execer` (swap in tests). |
|
||||
| `mcp.Render(name, servers)` | writes `{"mcpServers": ...}` to `$XDG_RUNTIME_DIR/sherlock/<name>.mcp.json`. |
|
||||
|
||||
If a new agent needs a third MCP-config schema, add a new `Render*`
|
||||
function to `internal/mcp/` rather than open-coding JSON in the agent
|
||||
file.
|
||||
|
||||
## What sherlock does *not* do
|
||||
|
||||
- Read agent config from `~/.config/sherlock/agents.d/` — that
|
||||
directory does not exist.
|
||||
- Hot-reload registered agents — the registry is sealed at process
|
||||
start, by design (one fewer code path).
|
||||
- Sandbox the agent — sherlock just `exec`s it, the agent inherits
|
||||
the user's full environment minus a few targeted forbids.
|
||||
Adding an agent is a code change under `internal/agent/`. Implement the small
|
||||
agent interface, register it in `init`, and use `internal/mcp` for config
|
||||
rendering. The exact API lives in `internal/agent/agent.go`; shared spawn
|
||||
helpers live in `internal/agent/exec.go`.
|
||||
|
||||
+23
-58
@@ -1,67 +1,32 @@
|
||||
# Architecture
|
||||
|
||||
Start here for the high-level picture; the other `docs/*.md` files cover each concern in depth.
|
||||
Sherlock is a local CLI, not a daemon. It either manages the wallet (`status`,
|
||||
`logout`) or replaces itself with an agent CLI (`copilot`, `claude`, ...). MCPs
|
||||
run as agent child processes and own service authentication.
|
||||
|
||||
## One-paragraph summary
|
||||
## Runtime flow
|
||||
|
||||
Sherlock is a per-user credential broker + agent-CLI wrapper that runs on the operator's workstation. It owns a single Authentik session (persisted in the OS keyring), exchanges it for per-service tokens on demand, injects those tokens as environment variables into thin stdio MCP servers, and then `exec`s the agent CLI of your choice (Copilot, Claude Code, …) with the right MCP config. No long-lived service tokens live on disk in the clear, and the agent never sees a credential it isn't supposed to.
|
||||
1. `sherlock <agent>` opens the keyring, resolves installed MCP binaries, writes
|
||||
`$XDG_RUNTIME_DIR/sherlock/<agent>.mcp.json`, and `exec`s the agent.
|
||||
2. The agent starts the configured stdio MCPs.
|
||||
3. Each MCP loads its `[services.<name>]` config, authenticates on first tool
|
||||
call, keeps its token fresh, and calls the target service.
|
||||
|
||||
## Diagram
|
||||
## Code map
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
user["user shell"] -->|`sherlock copilot`| sh[sherlock CLI]
|
||||
|
||||
sh -->|syscall.Exec| agent[copilot / claude CLI]
|
||||
sh -.->|writes MCP config| mcpfile[(XDG_RUNTIME_DIR/sherlock/<agent>.mcp.json)]
|
||||
agent -.->|reads| mcpfile
|
||||
|
||||
agent -->|stdio| giteamcp[gitea-mcp]
|
||||
agent -->|stdio| grafanamcp[grafana-mcp]
|
||||
agent -->|stdio| minifluxmcp[miniflux-mcp]
|
||||
|
||||
giteamcp -->|authn.Ensure| wallet[(OS keyring<br/>wallet: one TokenSet per service)]
|
||||
grafanamcp -->|authn.Ensure| wallet
|
||||
minifluxmcp -->|authn.Ensure| wallet
|
||||
sh -->|status / logout| wallet
|
||||
|
||||
wallet -->|OAuth PKCE flow on miss<br/>refresh on stale<br/>flock-serialised| authentik[Authentik<br/>gitea provider · grafana provider ·<br/>miniflux provider · …]
|
||||
```
|
||||
|
||||
Each MCP authenticates against the Authentik provider for its own
|
||||
service, lazily, the first time it runs. The wallet caches the
|
||||
result; subsequent runs are silent until the refresh window opens.
|
||||
There is no master session.
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Lives at | Owns |
|
||||
|---|---|---|
|
||||
| `sherlock` (CLI) | `cmd/sherlock/` | The only binary the operator runs. `status`, `logout [<service>]`, `run`, agent-name aliases (`copilot`, `claude`, …). At spawn: looks up the agent in `internal/agent/`, renders the per-session MCP config, `exec`s the agent. **There is no `sherlock login`** — see [auth-model.md](auth-model.md). |
|
||||
| `internal/agent/` | — | One Go file per supported CLI (`copilot.go`, `claude.go`, …), each registering itself via `init()`. Shared exec/env helpers live alongside. See [agents.md](agents.md). |
|
||||
| `internal/authn/` | — | OAuth/OIDC primitives + the high-level `Ensure(ctx, store, service, cfg, opts)` that every MCP calls at startup. See [auth-model.md](auth-model.md). |
|
||||
| `internal/keyring/` | — | OS keyring wallet, service-keyed: `Get` / `Set` / `Clear` / `List` per service. `Open()` is the only constructor; it probes the keyring and fails fast. See [storage.md](storage.md). |
|
||||
| `internal/mcp/` | — | Per-format MCP-config renderers (VS Code shape for Copilot, `.mcp.json` shape for Claude Code). |
|
||||
| `internal/xdg/` | — | Resolves `$XDG_RUNTIME_DIR/sherlock/<agent>.mcp.json` and the refresh-lock path. |
|
||||
| `cmd/gitea-mcp/` (Phase 2) | — | First per-service MCP. Reads service config from `~/.config/sherlock/services.d/gitea.toml`, calls `authn.Ensure(store, "gitea", cfg, opts)` at startup, then serves MCP requests. |
|
||||
| `cmd/gssh-mcp/` (Phase 3) | — | Thin HTTP+WS client to the existing gssh server. No local certs. See [gssh-integration.md](gssh-integration.md). |
|
||||
| `cmd/grafana-mcp/` | — | Imports upstream `mcp-grafana` as a Go package and serves its read-only tools in-process; injects a sherlock-renewed OAuth bearer per request. Requires Grafana `[auth.jwt]`. See [grafana-mcp.md](grafana-mcp.md). |
|
||||
| `cmd/sherlock-mcp/` (Phase 4) | — | The only **interactive** MCP. Exposes `auth.list_sessions()`, `auth.revoke(service)`, etc., so the agent can query/manage the wallet through tool calls. |
|
||||
|
||||
## Why there is no daemon, and no `sherlock login`
|
||||
|
||||
The Phase 1 design had a separate `sherlock-broker` daemon (forked-child, UDS RPC, PID-file flock, idle timer). It was removed in the post-Phase-1 refactor — Decisions #9 (JSON-over-newline RPC) and #10 (forked-child broker) are both superseded. Then `sherlock login` itself was removed in a follow-up refactor, because preemptive authentication never matched the actual topology.
|
||||
|
||||
Reasoning:
|
||||
|
||||
- **No daemon needed.** The OS keyring is already the single source of truth across processes. Cross-process refresh races are solved by an exclusive `flock()` on `$XDG_RUNTIME_DIR/sherlock.refresh.lock` with the canonical "take lock → re-read → maybe refresh" pattern in `internal/authn/refresh.go`. Concurrent *fresh logins* are serialised by a sibling `flock()` on `$XDG_RUNTIME_DIR/sherlock.login.lock` (`internal/authn/loginlock.go`) so they don't collide on the fixed loopback port.
|
||||
- **No master session.** Caddy + each downstream service have their own Authentik OAuth providers with their own audiences, scopes, and consent screens. A single master token cannot satisfy all of them. Per-service tokens are the right granularity, and per-service tokens are best fetched on demand by the MCP that needs them.
|
||||
- **No `sherlock login` step.** Pre-authenticating to N services up-front for the case where the user only ends up using one of them is wasted browser flows. Lazy-on-first-use is the right default; the wallet caches the rest.
|
||||
|
||||
What we lose: a small amount of latency overhead per refresh (libsecret D-Bus round-trip ~1ms) and the ability to handle a mid-session OAuth pop-up without a user click. What we gain: no daemon lifecycle, no PID files, no IPC protocol, no idle timers, no preemptive-login state machine, and "is sherlock-broker still running" stops being a debugging path.
|
||||
| Area | Role |
|
||||
| --------------------------------------------- | --------------------------------------------------------- |
|
||||
| `cmd/sherlock/` | CLI, wallet commands, agent dispatch, update entry point. |
|
||||
| `cmd/*-mcp/` | Service MCP binaries. Tool details live with each MCP. |
|
||||
| `internal/agent/` | Registered agent profiles and spawn helpers. |
|
||||
| `internal/mcp/` | MCP config rendering. |
|
||||
| `internal/config/` | TOML loading and service/provider resolution. |
|
||||
| `internal/authn/` | OAuth/PKCE login, refresh, token sources, locks. |
|
||||
| `internal/keyring/` | OS keyring wallet. |
|
||||
| `internal/installer/`, `internal/selfupdate/` | Install and update logic. |
|
||||
|
||||
## Boundaries
|
||||
|
||||
- Sherlock does **not** federate identities — that's Authentik's job.
|
||||
- Sherlock does **not** issue SSH certificates — gssh already does that internally when it authenticates a WebSocket session. See [gssh-integration.md](gssh-integration.md).
|
||||
- Sherlock does **not** know what tools an MCP exposes — that's the MCP's job. Sherlock only ensures the MCP has the credential it needs.
|
||||
Sherlock does not federate identities, run a background broker, issue SSH
|
||||
credentials, enforce service policy, or define MCP tool behavior. It gets the
|
||||
right credential to the right local MCP process.
|
||||
|
||||
+24
-116
@@ -1,128 +1,36 @@
|
||||
# Auth model
|
||||
|
||||
How sherlock obtains and distributes credentials.
|
||||
Sherlock is a service-keyed OAuth wallet. There is no `sherlock login` and no
|
||||
master session: each MCP authenticates for the service it calls.
|
||||
|
||||
## Mental model: wallet of OAuth sessions
|
||||
## Lifecycle
|
||||
|
||||
Sherlock is a wallet. Each entry is one OAuth session against one upstream
|
||||
service's Authentik provider (gitea, grafana, miniflux, invidious, …). There is
|
||||
**no master Authentik session**; every entry is service-keyed.
|
||||
For a service such as `gitea`, `grafana`, or `gssh`, the MCP asks
|
||||
`internal/authn` for a token on first use:
|
||||
|
||||
This matches the homelab topology: caddy reverse-proxies the public endpoints
|
||||
and enforces edge OIDC for browsers; each downstream service *also* has its own
|
||||
Authentik OAuth client used to bind a service-side account to the operator's
|
||||
Authentik identity. An MCP that wants to call `gitea.x` needs a token issued by
|
||||
gitea's specific Authentik provider — caddy + gitea both verify it. The same
|
||||
goes for every other service. One master token cannot satisfy all of them.
|
||||
1. Fresh wallet entry: return it.
|
||||
2. Stale entry with refresh token: refresh under
|
||||
`$XDG_RUNTIME_DIR/sherlock.refresh.lock`, persist, return.
|
||||
3. Missing or unrecoverable entry: run a PKCE browser flow on `127.0.0.1:6990`,
|
||||
serialized by `$XDG_RUNTIME_DIR/sherlock.login.lock`, persist, return.
|
||||
|
||||
## Authentication is lazy and per-MCP
|
||||
Long-running MCPs use `TokenSource` and `TokenHolder` so requests always read
|
||||
the latest bearer. Refresh never opens a browser; if refresh can no longer
|
||||
recover, the next invocation performs a fresh login.
|
||||
|
||||
`sherlock login` does not exist. The operator does not preemptively
|
||||
authenticate. Each MCP at startup calls
|
||||
## Service identity
|
||||
|
||||
```go
|
||||
ts, err := authn.Ensure(ctx, store, "gitea", cfg, opts)
|
||||
```
|
||||
The config supplies issuer, client ID/secret, and base URL. Scopes are owned by
|
||||
each MCP because they follow the tool surface, not the deployment.
|
||||
|
||||
`Ensure` (in `internal/authn`) handles every case:
|
||||
Gitea uses Gitea's OAuth2 server. Grafana and Gssh normally reuse the shared
|
||||
Authentik `sherlock-cli` provider. Tokens are stored and refreshed per service
|
||||
and are not reused across unrelated services.
|
||||
|
||||
1. Wallet has fresh tokens for `gitea` → return them.
|
||||
2. Wallet has stale tokens with a usable refresh token → refresh under a
|
||||
cross-process flock, persist, return.
|
||||
3. Wallet is empty (or refresh failed) → bind the loopback listener on
|
||||
`127.0.0.1:6990`, run a fresh OAuth PKCE flow against the service's Authentik
|
||||
provider (opens a browser via `xdg-open`), persist, return.
|
||||
## User controls
|
||||
|
||||
The first time an operator runs `sherlock copilot` in a fresh environment, the
|
||||
agent spawns its MCPs; each MCP triggers its own browser flow as it needs one.
|
||||
Subsequent runs are silent until the refresh window opens.
|
||||
`sherlock status` lists stored sessions. `sherlock logout` clears all sessions;
|
||||
`sherlock logout <service>` clears one.
|
||||
|
||||
### Concurrent first-use is serialised
|
||||
|
||||
Because the loopback port (`127.0.0.1:6990`) is a single machine-wide resource,
|
||||
two MCPs hitting their first tool call at the same moment would otherwise both
|
||||
try to bind it and one would fail with `EADDRINUSE`. `Ensure` wraps the fresh
|
||||
login in an exclusive flock on `$XDG_RUNTIME_DIR/sherlock.login.lock` (sibling of
|
||||
the refresh lock, but separate so a long, human-in-the-loop login never blocks a
|
||||
quick refresh). The flows queue; after the first one authenticates, the rest
|
||||
reuse the operator's existing Authentik browser session and complete without a
|
||||
second click. Lock acquisition is context-cancellable, so a waiting flow gives up
|
||||
cleanly if its request is cancelled. See `internal/authn/loginlock.go`.
|
||||
|
||||
## Token renewal
|
||||
|
||||
Authentik access tokens are short-lived (the `sherlock-cli` provider issues
|
||||
~5-minute tokens). An MCP that captured a token once at startup would start
|
||||
getting 401s a few minutes into a long agent session. So MCPs do not hold a
|
||||
static token — they hold a renewer.
|
||||
|
||||
`authn.TokenSource` owns the refresh loop; the MCP supplies a **mandatory
|
||||
callback** that the source invokes with every new token:
|
||||
|
||||
```go
|
||||
holder := authn.NewTokenHolder()
|
||||
src := authn.NewTokenSource(store, "grafana", cfg, opts, holder.Set)
|
||||
|
||||
ts, err := src.Start(ctx) // initial OAuth (browser on first use); calls holder.Set
|
||||
go src.Run(ctx) // renews ahead of expiry; calls holder.Set on every rotation
|
||||
```
|
||||
|
||||
- `Start(ctx)` does the initial `Ensure` and pushes the first token to the
|
||||
callback.
|
||||
- `Run(ctx)` sleeps until `RefreshSkew` before the token expires, refreshes
|
||||
under the same cross-process flock as `EnsureFresh`, and — when the access
|
||||
token actually rotates — invokes the callback again. It is refresh-only and
|
||||
never opens a browser; if the refresh token itself has expired it backs off
|
||||
and leaves re-login to the next `sherlock` invocation.
|
||||
- The callback is required (a nil callback panics at construction). The request
|
||||
path never polls: it reads the current bearer from the `TokenHolder` the
|
||||
callback writes (`holder.AccessToken` / `holder.Bearer`), which is lock-free
|
||||
and always reflects the latest renewal.
|
||||
|
||||
This is why every MCP's API client takes a token *getter*, not a token string:
|
||||
`grafana-mcp` reads it from a per-request `BaseTransport`, `gitea-mcp` and
|
||||
`gssh-mcp` from `holder.AccessToken` on each REST call and WebSocket dial.
|
||||
|
||||
## What lives in `services.d/<name>.toml` (Phase 2)
|
||||
|
||||
Each MCP needs an OIDC `Config` (issuer + client ID + scopes) to pass to
|
||||
`Ensure`. The operator-editable source of truth is
|
||||
|
||||
```toml
|
||||
[service]
|
||||
name = "gitea"
|
||||
issuer = "https://id.alexandru.macocian.me/application/o/gitea/"
|
||||
client_id = "Ig8...abc"
|
||||
scopes = ["openid", "profile", "email"]
|
||||
audience = "gitea.alexandru.macocian.me" # informational; aud check is on the IdP side
|
||||
|
||||
[mcp]
|
||||
env_var = "GITEA_TOKEN" # what sherlock injects into the agent env (Phase 2+)
|
||||
```
|
||||
|
||||
Sherlock-the-CLI does not read these in Phase 1 (the CLI has no authentication
|
||||
subcommands). Phase 2 MCPs load their own service's TOML at startup. See
|
||||
[service-registry.md](service-registry.md) for the full schema.
|
||||
|
||||
## Audience binding
|
||||
|
||||
Per the MCP 2025-06-18 spec, downstream tokens MUST be audience-bound. Each
|
||||
Authentik provider issues tokens with `aud` set to its service's canonical name.
|
||||
Caddy and the upstream both verify it. Sherlock never tries to reuse one
|
||||
service's token for another.
|
||||
|
||||
## Scope minimisation
|
||||
|
||||
Default service registrations request read-only scopes. Write scopes require an
|
||||
explicit override in the service's TOML (Phase 2).
|
||||
|
||||
## Out of scope for sherlock
|
||||
|
||||
- Issuing SSH certs (gssh handles this —
|
||||
[gssh-integration.md](gssh-integration.md)).
|
||||
- Multi-user / shared tenancy. Sherlock is per-operator, full stop.
|
||||
- Acting as a Resource Server itself.
|
||||
- RFC 8693 token exchange. Tried during planning; rejected because each
|
||||
service's Authentik provider has its own consent screen and scope set and the
|
||||
exchange story across them is more brittle than just running N separate PKCE
|
||||
flows lazily.
|
||||
Sherlock does not use static service-account tokens, share sessions between
|
||||
operators, or perform RFC 8693 token exchange.
|
||||
|
||||
+20
-73
@@ -1,89 +1,36 @@
|
||||
# Configuration
|
||||
|
||||
Sherlock reads all deployment-specific values — the Authentik
|
||||
issuer/client IDs and each service's base URL — from a single TOML file.
|
||||
There are **no compiled-in defaults**: a missing file, section, or
|
||||
required field is a hard error, so an MCP can never silently target the
|
||||
wrong host.
|
||||
Sherlock reads deployment values from one TOML file. Missing files, sections, or
|
||||
required fields are hard errors.
|
||||
|
||||
A starter file lives at [`config.example.toml`](../config.example.toml).
|
||||
Default source: [`config.example.toml`](../config.example.toml).
|
||||
|
||||
## Location
|
||||
|
||||
Resolved in this order:
|
||||
Resolved in order:
|
||||
|
||||
1. `$SHERLOCK_CONFIG` (an explicit path; handy for tests / non-standard
|
||||
installs).
|
||||
2. `$XDG_CONFIG_HOME/sherlock/config.toml`.
|
||||
3. `$HOME/.config/sherlock/config.toml`.
|
||||
1. `$SHERLOCK_CONFIG`
|
||||
2. `$XDG_CONFIG_HOME/sherlock/config.toml`
|
||||
3. `$HOME/.config/sherlock/config.toml`
|
||||
|
||||
## Shape
|
||||
|
||||
```toml
|
||||
[providers.sherlock-cli]
|
||||
issuer = "https://id.example.com/application/o/sherlock-cli/"
|
||||
client_id = "…"
|
||||
`[providers.<name>]` defines a reusable OAuth/OIDC identity: `issuer`,
|
||||
`client_id`, optional `client_secret`.
|
||||
|
||||
[services.gitea]
|
||||
issuer = "https://gitea.example.com"
|
||||
client_id = "…"
|
||||
base_url = "https://gitea.example.com"
|
||||
`[services.<name>]` defines one MCP target. The service name is also the wallet
|
||||
key used by `sherlock status` and `sherlock logout <name>`. Each service
|
||||
requires `base_url` plus either:
|
||||
|
||||
[services.grafana]
|
||||
provider = "sherlock-cli"
|
||||
base_url = "https://grafana.example.com"
|
||||
- `provider = "<name>"`, resolved from `[providers.<name>]`, or
|
||||
- inline `issuer`, `client_id`, and optional `client_secret`.
|
||||
|
||||
[services.gssh]
|
||||
provider = "sherlock-cli"
|
||||
base_url = "https://terminal.example.com"
|
||||
```
|
||||
Mixing `provider` with inline identity fields is rejected.
|
||||
|
||||
### Providers
|
||||
## Not configured here
|
||||
|
||||
A `[providers.<name>]` block is a reusable OAuth/OIDC client
|
||||
(`issuer`, `client_id`, optional `client_secret`). Define it once and
|
||||
share it across every service it fronts. The `sherlock-cli` Authentik
|
||||
provider is shared by `grafana` and `gssh` (and any future
|
||||
Authentik-fronted service); it is a Public/PKCE client, so
|
||||
`client_secret` is omitted and its redirect URI must be whitelisted as
|
||||
`http://127.0.0.1:6990/callback`.
|
||||
Tokens live in the OS keyring. OAuth scopes live in MCP code. Agent profiles and
|
||||
the built-in MCP registry live in Go code.
|
||||
|
||||
### Services
|
||||
|
||||
Each `[services.<name>]` is one MCP's target. `<name>` is both the
|
||||
config key and the wallet key (`sherlock logout <name>`). A service
|
||||
must resolve to a non-empty `issuer`, `client_id`, and `base_url`, by
|
||||
either:
|
||||
|
||||
- **referencing a provider** — `provider = "sherlock-cli"` (Authentik
|
||||
services), or
|
||||
- **carrying an inline identity** — `issuer` + `client_id`
|
||||
(+ `client_secret` if required). Gitea uses this form because it runs
|
||||
its own OAuth2 server rather than authenticating against Authentik.
|
||||
|
||||
Setting both `provider` and an inline `issuer`/`client_id` on the same
|
||||
service is rejected — pick one.
|
||||
|
||||
`base_url` is the API origin the MCP calls (e.g. `/api/v1/...` for
|
||||
Gitea, `/api/...` for Grafana). It is always required.
|
||||
|
||||
## What is *not* in the config
|
||||
|
||||
- **OAuth scopes** — each MCP declares the scopes it needs in code
|
||||
(Gitea pulls a broad read set; Grafana/Gssh ask for
|
||||
`openid profile email`). Scopes are a function of what the MCP does,
|
||||
not of the deployment.
|
||||
- **Secrets / tokens** — the operator's OAuth tokens live in the OS
|
||||
keyring, never in this file. See [storage.md](storage.md).
|
||||
- **The loopback redirect port** (`127.0.0.1:6990`) — fixed in code;
|
||||
see [auth-model.md](auth-model.md).
|
||||
|
||||
## Adding a service
|
||||
|
||||
1. Add a `[services.<name>]` block (provider reference or inline
|
||||
identity, plus `base_url`).
|
||||
2. Point its MCP's `serviceName` constant at `<name>` (it's the config
|
||||
key and the wallet key).
|
||||
|
||||
See [conventions.md](conventions.md) for the broader extensibility
|
||||
model.
|
||||
To add a service, add its config entry and wire or install the matching MCP
|
||||
binary.
|
||||
|
||||
+30
-49
@@ -1,68 +1,49 @@
|
||||
# Conventions
|
||||
|
||||
Rules that govern this repo. Anything that becomes load-bearing belongs here, in `conventions.md`, never in a section appended to another doc.
|
||||
Repository rules that should stay true as the project changes.
|
||||
|
||||
## Repository layout
|
||||
## Layout
|
||||
|
||||
```
|
||||
cmd/ one subdir per shippable binary; package main only
|
||||
setup/ bootstrap installer entry (go run ./setup); not shipped
|
||||
internal/ every other package; never imported outside this module
|
||||
internal/installer/ the one install/update implementation (shared by setup + `sherlock update`)
|
||||
docs/ one topic per file (see "Docs" below)
|
||||
VERSION release version source of truth (see versioning.md)
|
||||
README.md TOC only
|
||||
```
|
||||
|
||||
No top-level `pkg/` until we have an external consumer.
|
||||
- Shippable binaries live under `cmd/<name>/`.
|
||||
- The bootstrap installer lives under `setup/`; shared install logic lives in
|
||||
`internal/installer/`.
|
||||
- Shared Go code lives under `internal/`. Add no top-level `pkg/` until there is
|
||||
an external consumer.
|
||||
- `README.md` is a short description plus a docs index.
|
||||
- `VERSION` is the release version source of truth.
|
||||
|
||||
## Go
|
||||
|
||||
- Module path: `gitea.alexandru.macocian.me/amacocian/sherlock`.
|
||||
- Target Go toolchain: `go 1.25` (pinned via `go.mod`'s `go` directive; CI installs the matching minor).
|
||||
- One `package main` per `cmd/<binary>/`. No multiple-`main`-files trickery.
|
||||
- Every other package has a top-of-file `Package <name> ...` doc comment (in the leading `.go` source file; we drop standalone `doc.go` files once a real source file exists).
|
||||
- Third-party deps are added with a one-line justification. The current set:
|
||||
- `github.com/zalando/go-keyring` — OS keyring for token persistence.
|
||||
- `github.com/BurntSushi/toml` — parse the operator config (`config.toml`).
|
||||
- `github.com/coreos/go-oidc/v3` — OIDC discovery + ID-token verification against Authentik.
|
||||
- `golang.org/x/oauth2` — auth-code + PKCE + refresh against Authentik's token endpoint.
|
||||
- `golang.org/x/mod` — `semver` for version comparison in self-update.
|
||||
- `github.com/grafana/mcp-grafana` + `github.com/mark3labs/mcp-go` — upstream Grafana MCP tool set, served in-process by `cmd/grafana-mcp`.
|
||||
- `github.com/modelcontextprotocol/go-sdk` — MCP server SDK for the gitea/gssh MCPs.
|
||||
- `github.com/coder/websocket` — gssh exec WebSocket client.
|
||||
- Target Go version: the `go` directive in `go.mod`.
|
||||
- One `package main` per `cmd/<binary>/`.
|
||||
- Non-main packages have a package doc comment in their leading source file.
|
||||
- New third-party dependencies need a short justification with the change.
|
||||
|
||||
## Docs
|
||||
|
||||
- `README.md` is **TOC only**. One-line description + a bulleted index of `docs/*.md`.
|
||||
- **One topic per file under `docs/`.** Never append a new section to an existing doc to cover a new concern; create `docs/<new-topic>.md` and link it from `README.md`.
|
||||
- Cross-link aggressively: every doc should link to the other docs whose concerns it touches.
|
||||
- Mermaid diagrams over images or ASCII art. Diagrams live next to the prose that explains them.
|
||||
- Keep docs short and functional.
|
||||
- Prefer links to source files over copied API/tool examples.
|
||||
- Do not add layout diagrams. Use prose for relationships and source links for
|
||||
details.
|
||||
- One topic per `docs/*.md` file; link from `README.md`.
|
||||
|
||||
## Naming
|
||||
|
||||
- Binaries are kebab-case (`gitea-mcp`, `gssh-mcp`).
|
||||
- Built-in agent profiles match the agent's canonical CLI name (`copilot`, `claude`, `aider`).
|
||||
- Service registry files match the service's canonical name (`gitea.toml`, `grafana.toml`).
|
||||
- Env vars are `<SERVICE>_TOKEN` (e.g. `GITEA_TOKEN`, `GSSH_TOKEN`).
|
||||
- Binaries are kebab-case: `gitea-mcp`, `gssh-mcp`.
|
||||
- Agent names match the wrapped CLI: `copilot`, `claude`.
|
||||
- Service names are the config key and wallet key: `gitea`, `grafana`, `gssh`.
|
||||
|
||||
## Extensibility invariants
|
||||
## Commits and CI
|
||||
|
||||
- **Agent extensibility:** adding a new agent CLI is a Go file under `internal/agent/` registering itself via `init()`. See [agents.md](agents.md). The TOML-overlay design was tried and dropped — per-CLI quirks (auth subcommands, flag schemas, MCP config shapes) deserve a real code home.
|
||||
- **Service extensibility:** adding a new downstream service is a TOML drop-in under `~/.config/sherlock/services.d/` + (optionally) a new `cmd/<service>-mcp/` binary. Sherlock's own code does not learn about individual services.
|
||||
|
||||
## Commits
|
||||
|
||||
- Conventional-ish: `area: short imperative` (e.g. `authn: persist client_id in TokenSet`, `docs: add gssh-integration`).
|
||||
- One logical change per commit. CI must pass on every commit on `main`.
|
||||
|
||||
## CI
|
||||
|
||||
- `.gitea/workflows/release.yaml` is the single pipeline. On every push + PR it runs `gofmt`, `go vet`, `errcheck`, `staticcheck`, `go test -race`, and `go build`.
|
||||
- On **push to `main` only**, and **only after every gate above passes**, a final step reads the root `VERSION` file and pushes the tag `vVERSION` if it doesn't already exist. A tag is never created on a red build. Cutting a release is a one-line edit to `VERSION`. See [versioning.md](versioning.md).
|
||||
- Sherlock is operator-installed via `install.sh` (which clones + `go install`s) and self-updates via `sherlock update`. No host-deploy pipeline.
|
||||
- Commit style: `area: short imperative`.
|
||||
- Keep one logical change per commit.
|
||||
- `.gitea/workflows/release.yaml` runs formatting, static analysis, race tests,
|
||||
and build.
|
||||
- Tags are created from `VERSION` only after the main-branch release workflow
|
||||
passes.
|
||||
|
||||
## Security
|
||||
|
||||
- No secrets in this repo, ever. Not in tests, not in fixtures, not in comments.
|
||||
- The operator's Authentik tokens and (Phase 2+) per-service tokens live in the OS keyring. See [storage.md](storage.md).
|
||||
No secrets in the repository. OAuth tokens live in the OS keyring; config files
|
||||
contain deployment metadata only.
|
||||
|
||||
+21
-255
@@ -1,264 +1,30 @@
|
||||
# gitea-mcp
|
||||
|
||||
The first sherlock MCP. Wraps a small slice of the Gitea REST API
|
||||
(`whoami`, `list_repos`) using a Gitea-issued OAuth2 bearer token.
|
||||
Stdio MCP for Gitea's REST API. It exposes read-only access to user,
|
||||
repository/content, refs, commits, issues, pull requests, releases, wiki,
|
||||
actions, packages, and organisation data. Exact tool names and schemas live in
|
||||
`cmd/gitea-mcp/tools_*.go`.
|
||||
|
||||
## How auth works
|
||||
## Auth
|
||||
|
||||
Gitea has two relevant features that share the word "OAuth":
|
||||
`gitea-mcp` uses Gitea's OAuth2 server with PKCE and stores its session under
|
||||
wallet key `gitea`. Gitea web SSO through Authentik is incidental; API tokens
|
||||
are minted by Gitea.
|
||||
|
||||
1. **Gitea as OAuth *client*** — Gitea logs users into its web UI via
|
||||
Authentik (your homelab SSO). This is how human accounts get
|
||||
provisioned the first time someone visits Gitea.
|
||||
2. **Gitea as OAuth *server*** — Gitea is itself an OAuth 2.0 / OIDC
|
||||
provider that third-party apps can authenticate against. The
|
||||
resulting access token is accepted by Gitea's own REST API.
|
||||
Config is `[services.gitea]` with inline `issuer`, `client_id`, optional
|
||||
`client_secret`, and `base_url`. Scopes are compiled in `cmd/gitea-mcp/main.go`;
|
||||
after scope changes, clear the wallet entry with `sherlock logout gitea`.
|
||||
|
||||
`gitea-mcp` uses (2). The flow does not touch Authentik at all from
|
||||
sherlock's perspective — sherlock OAuths directly against Gitea using
|
||||
PKCE, gets a Gitea-minted bearer token, and uses it as
|
||||
`Authorization: Bearer <token>` on `/api/v1/...` calls. The fact that
|
||||
Gitea behind the scenes might bounce you through Authentik for SSO is
|
||||
invisible to sherlock.
|
||||
## Operation
|
||||
|
||||
This is deliberate. Going through Authentik would require Gitea to
|
||||
trust Authentik-issued bearer tokens at its API layer, which Gitea
|
||||
doesn't do out of the box. Going through Gitea's own OAuth2 server is
|
||||
the documented, supported path.
|
||||
When `gitea-mcp` is installed, `sherlock <agent>` includes it in the generated
|
||||
MCP config. The first tool call authenticates lazily; later calls use refreshed
|
||||
keyring tokens. `gitea-mcp --probe` verifies auth and one API call without an
|
||||
agent.
|
||||
|
||||
### Known Gitea scope limitation
|
||||
## Known limitation
|
||||
|
||||
Gitea's OAuth2 server does **not** honour the full scope list sherlock
|
||||
requests. We send `openid profile email read:user read:repository
|
||||
read:issue read:organization read:package`, but Gitea silently grants
|
||||
only `read:repository read:user` regardless. Verified end-to-end
|
||||
2026-05-28 against `gitea.alexandru.macocian.me`: the token's `scope`
|
||||
attribute on `/login/oauth/access_token` comes back narrowed, and any
|
||||
subsequent call into an ungranted area returns:
|
||||
|
||||
```
|
||||
HTTP 403: token does not have at least one of required scope(s),
|
||||
required=[read:issue|read:organization|read:package],
|
||||
token scope=read:repository,read:user
|
||||
```
|
||||
|
||||
Affected tools (every one returns 403 with this token, regardless of
|
||||
how many times you `sherlock logout gitea` and re-auth):
|
||||
|
||||
- **Issues / PRs:** `list_issues`, `get_issue`, `list_issue_comments`
|
||||
(PRs themselves go through `read:repository` and *do* work).
|
||||
- **Organisations:** `list_my_orgs`, `get_org`, `list_org_repos`,
|
||||
`list_org_members`, `list_org_teams`, `search_org_teams`,
|
||||
`list_org_activity_feed`, `list_org_workflow_runs`,
|
||||
`list_org_workflow_jobs`, `list_org_runners`, `get_org_runner`.
|
||||
- **Packages:** `list_packages`.
|
||||
|
||||
This is a Gitea-side issue (the OAuth2 server doesn't surface those
|
||||
scopes on the consent screen, so the user can't grant them even if
|
||||
they wanted to). Workarounds we may consider later:
|
||||
|
||||
1. Use a long-lived PAT scoped explicitly to issue/org/package read
|
||||
instead of an OAuth token for these tool families (would require a
|
||||
second wallet entry and a `--use-pat` knob on the MCP).
|
||||
2. Patch / upgrade Gitea once upstream wires these scopes into the
|
||||
OAuth2 consent flow.
|
||||
3. Drop the affected tools from the surface and document the gap.
|
||||
|
||||
Until one of those lands, those tools stay registered but will 403 at
|
||||
call time. The MCP itself does not refuse to start.
|
||||
|
||||
## One-time setup
|
||||
|
||||
1. Sign in to https://gitea.alexandru.macocian.me as the operator.
|
||||
2. **Settings → Applications → Manage OAuth2 Applications → New
|
||||
Application**:
|
||||
- **Application Name:** `sherlock`
|
||||
- **Redirect URIs:** `http://127.0.0.1:6990/callback`
|
||||
- **Confidential Client:** ⬜ **UNCHECK** — sherlock uses PKCE,
|
||||
not a client secret. Leaving this checked makes Gitea demand a
|
||||
`client_secret` on the token request, which sherlock never sends.
|
||||
3. Click **Create Application**.
|
||||
4. Copy the **Client ID** Gitea displays.
|
||||
|
||||
## Build & install
|
||||
|
||||
For Charlie (default — Client ID embedded in the binary):
|
||||
|
||||
```bash
|
||||
cd ~/Dev/charlie/sherlock
|
||||
go install ./cmd/gitea-mcp
|
||||
```
|
||||
|
||||
That's it. No `-ldflags`, no flags at all. The Charlie Gitea OAuth2
|
||||
app's Client ID lives in `cmd/gitea-mcp/gitea_clientid_charlie.go`
|
||||
and is baked in unless you build with `-tags noembed`.
|
||||
|
||||
For other deployments:
|
||||
|
||||
```bash
|
||||
go build -tags noembed \
|
||||
-ldflags "-X main.giteaIssuer=https://gitea.example.com \
|
||||
-X main.giteaClientID=<CLIENT_ID> \
|
||||
-X main.giteaBaseURL=https://gitea.example.com" \
|
||||
./cmd/gitea-mcp
|
||||
```
|
||||
|
||||
Modern Gitea (≥ 1.16-ish) accepts PKCE-alone for OAuth2 applications
|
||||
that have the "Confidential Client" checkbox unchecked, so no client
|
||||
secret is needed even though Gitea generates one in the UI.
|
||||
|
||||
### Knobs
|
||||
|
||||
| Variable / `-X main.…` | Charlie default |
|
||||
|------------------------|-----------------------------------------------|
|
||||
| `giteaIssuer` | `https://gitea.alexandru.macocian.me` |
|
||||
| `giteaBaseURL` | `https://gitea.alexandru.macocian.me` |
|
||||
| `giteaClientID` | embedded; see `gitea_clientid_charlie.go` |
|
||||
| `giteaClientSecret` | `""` (only needed for older / confidential Gitea apps) |
|
||||
| `Version` | `0.0.0-dev` |
|
||||
|
||||
When pointing at a different Gitea deployment, override `giteaIssuer`,
|
||||
`giteaBaseURL`, and `giteaClientID` together (with `-tags noembed`).
|
||||
Sherlock treats the wallet entry as service `gitea` regardless of
|
||||
which deployment the token came from, so don't mix.
|
||||
|
||||
> If `gitea-mcp --probe` ever hits `invalid_client` or similar after
|
||||
> you click Authorize in the browser, your Gitea is enforcing client
|
||||
> auth on token exchange. Rebuild with
|
||||
> `-X main.giteaClientSecret=<the secret>` to include it. The secret
|
||||
> is baked into the local binary, never written to the source tree.
|
||||
|
||||
## Verify (without an agent)
|
||||
|
||||
```bash
|
||||
gitea-mcp --probe
|
||||
```
|
||||
|
||||
First run: opens a browser, you click through Gitea's "Authorize
|
||||
sherlock" page (which itself goes through Authentik SSO if you're not
|
||||
already signed in), browser shows "Logged in. You may close this
|
||||
tab.", terminal prints:
|
||||
|
||||
```
|
||||
OK: logged in to https://gitea.alexandru.macocian.me as <login> <<email>>
|
||||
```
|
||||
|
||||
Subsequent runs are silent until the refresh window opens.
|
||||
|
||||
To force a re-login: `sherlock logout gitea`.
|
||||
|
||||
## Use with an agent
|
||||
|
||||
`sherlock copilot` (or `sherlock claude`) automatically renders an MCP
|
||||
config that lists `gitea-mcp`. Copilot spawns it as a stdio
|
||||
subprocess; the first call into it triggers the OAuth flow described
|
||||
above (browser pops while you're in the middle of a chat — accept
|
||||
once, never again).
|
||||
|
||||
Tools exposed (all read-only; write tools land in a follow-up):
|
||||
|
||||
### User
|
||||
|
||||
| Tool | Description |
|
||||
|---------------------|-------------------------------------------------|
|
||||
| `whoami` | Authenticated Gitea user. |
|
||||
|
||||
### Repos & content
|
||||
|
||||
| Tool | Description |
|
||||
|---------------------|-------------------------------------------------|
|
||||
| `list_repos` | Search/list repos accessible to the user. |
|
||||
| `get_file` | Read file contents (≤ 256 KiB, truncated tail). |
|
||||
| `list_dir` | List one directory level at a given ref. |
|
||||
| `get_tree` | Recursive listing of files/dirs (≤ 2000 entries) at a given ref. |
|
||||
| `file_history` | Commits touching a specific file. |
|
||||
| `list_branches` | Branches of a repo. |
|
||||
| `get_branch` | One branch's details + tip SHA. |
|
||||
| `list_tags` | Tags of a repo. |
|
||||
| `get_tag` | One tag's details. |
|
||||
|
||||
### Commits
|
||||
|
||||
| Tool | Description |
|
||||
|---------------------|-------------------------------------------------|
|
||||
| `list_commits` | Repo commit history with date / SHA filters. |
|
||||
| `get_commit` | Single commit by SHA (full message, parents). |
|
||||
| `get_commit_status` | Combined CI / status check state for a ref. |
|
||||
|
||||
### Issues
|
||||
|
||||
| Tool | Description |
|
||||
|-----------------------|-----------------------------------------------|
|
||||
| `list_issues` | Issues with state/labels/type filters. |
|
||||
| `get_issue` | One issue or PR by index. |
|
||||
| `list_issue_comments` | Comments on a single issue / PR. |
|
||||
|
||||
### Pull requests
|
||||
|
||||
| Tool | Description |
|
||||
|---------------------|-------------------------------------------------|
|
||||
| `list_pulls` | PRs with state/sort/labels filters. |
|
||||
| `get_pull` | Full PR details incl. requested reviewers. |
|
||||
| `list_pull_files` | Files changed by a PR (counts; no diffs). |
|
||||
| `list_pull_reviews` | Reviews on a PR (state, body, reviewer). |
|
||||
|
||||
### Releases
|
||||
|
||||
| Tool | Description |
|
||||
|---------------------|-------------------------------------------------|
|
||||
| `list_releases` | Releases of a repo (drafts & pre-releases opt-in). |
|
||||
| `get_release` | One release by ID, incl. assets. |
|
||||
|
||||
### Actions (CI)
|
||||
|
||||
| Tool | Description |
|
||||
|--------------------------|--------------------------------------------|
|
||||
| `list_workflow_runs` | Per-repo pipeline runs. |
|
||||
| `list_workflow_jobs` | Jobs of a specific run. |
|
||||
| `get_job_logs` | Tail (100 KiB) of a job's log output. |
|
||||
| `list_org_workflow_runs` | All runs across an org's repos. |
|
||||
| `list_org_workflow_jobs` | All jobs across an org's runs. |
|
||||
| `list_org_runners` | Self-hosted runners registered to an org. |
|
||||
| `get_org_runner` | Details for a single org runner. |
|
||||
|
||||
### Packages
|
||||
|
||||
| Tool | Description |
|
||||
|---------------------|-------------------------------------------------|
|
||||
| `list_packages` | Packages owned by a user/org (type + name). |
|
||||
|
||||
### Wiki
|
||||
|
||||
| Tool | Description |
|
||||
|---------------------|-------------------------------------------------|
|
||||
| `list_wiki_pages` | Wiki page titles/slugs/last-update. |
|
||||
| `get_wiki_page` | One wiki page content. |
|
||||
|
||||
### Organisations
|
||||
|
||||
| Tool | Description |
|
||||
|--------------------------|--------------------------------------------|
|
||||
| `list_my_orgs` | Orgs the authenticated user is a member of.|
|
||||
| `get_org` | One org's details. |
|
||||
| `list_org_repos` | Repos owned by an org. |
|
||||
| `list_org_members` | Members of an org. |
|
||||
| `list_org_teams` | Teams of an org. |
|
||||
| `search_org_teams` | Search teams in an org by name substring. |
|
||||
| `list_org_activity_feed` | Recent activity feed of an org. |
|
||||
|
||||
Write operations and admin-scoped tools (admin runners, admin
|
||||
workflow runs, admin org management) land in a separate
|
||||
`gitea-mcp-admin` MCP later so the per-tool permission surface stays
|
||||
small.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause |
|
||||
|------------------------------------------------------|---------------------------------------------------------------------------------------------|
|
||||
| `gitea-mcp: giteaClientID not configured.` | Built without `-ldflags "-X main.giteaClientID=..."`. See above. |
|
||||
| Browser opens, Gitea says **"Client ID not registered"** | Either the OAuth2 application was never created in Gitea, or the Client ID was mistyped. |
|
||||
| Browser opens, Gitea says **"client_secret required"** | The OAuth2 application has "Confidential Client" checked. Edit it and uncheck. |
|
||||
| `gitea: 401 (token rejected ...)` | Token was revoked or the wallet has stale tokens for a different Gitea deployment. Run `sherlock logout gitea` and retry. |
|
||||
| `bind: address already in use` | A previous OAuth flow (or another MCP) still holds `127.0.0.1:6990`. Wait a few seconds or kill the stuck process. |
|
||||
EOF
|
||||
echo "gitea-mcp doc created"
|
||||
Some Gitea versions narrow OAuth grants to `read:user` and `read:repository`.
|
||||
Tools that require issue, organisation, or package scopes can return 403 even
|
||||
though the MCP starts correctly. That is a Gitea-side scope grant issue, not a
|
||||
sherlock startup failure.
|
||||
|
||||
+17
-128
@@ -1,136 +1,25 @@
|
||||
# grafana-mcp
|
||||
|
||||
Sherlock's Grafana MCP **imports Grafana Labs' upstream
|
||||
[`mcp-grafana`](https://github.com/grafana/mcp-grafana) as a Go
|
||||
package** and serves a read-only subset of its tools in-process. There
|
||||
is no separate `mcp-grafana` binary, no `uvx`, and no `exec`.
|
||||
Read-only Grafana MCP. Sherlock imports Grafana Labs' upstream `mcp-grafana` as
|
||||
a Go package and registers the read-only search, datasource, Prometheus, Loki,
|
||||
alerting, dashboard, folder, navigation, and annotation categories. Exact tool
|
||||
behavior belongs to upstream and `cmd/grafana-mcp/main.go`.
|
||||
|
||||
It authenticates as the operator the same way `gitea-mcp` and
|
||||
`gssh-mcp` do — OAuth + PKCE against Authentik, token stored in the OS
|
||||
keyring — so the agent never sees a Grafana service-account token.
|
||||
## Auth
|
||||
|
||||
## How auth works
|
||||
`grafana-mcp` uses sherlock-managed OAuth with the wallet key `grafana`. It
|
||||
sends a fresh Authentik bearer on every Grafana API request and does not use
|
||||
`GRAFANA_SERVICE_ACCOUNT_TOKEN`, `GRAFANA_API_KEY`, or username/password auth.
|
||||
|
||||
`grafana-mcp` does **not** use `GRAFANA_SERVICE_ACCOUNT_TOKEN`,
|
||||
`GRAFANA_API_KEY`, or Grafana username/password auth.
|
||||
Grafana must accept external JWT bearers on its API through `[auth.jwt]`.
|
||||
Generic OAuth is only browser SSO and is not enough for `/api/*` bearer
|
||||
validation.
|
||||
|
||||
At startup it calls `authn.NewTokenSource(...).Start(ctx)` to obtain an
|
||||
Authentik access token (browser flow on first use). Every Grafana API
|
||||
request then carries that token as `Authorization: Bearer <jwt>`,
|
||||
injected by a custom `GrafanaConfig.BaseTransport`. The token is kept
|
||||
fresh by the sherlock `TokenSource` renewer (see
|
||||
[auth-model.md](auth-model.md#token-renewal)) — Authentik access tokens
|
||||
live only ~5 minutes, so a captured-once token would 401 mid-session.
|
||||
## Operation
|
||||
|
||||
Only read-only tool categories are registered (search, datasource,
|
||||
prometheus, loki, alerting, dashboard, folder, navigation, annotations);
|
||||
upstream write tools are disabled.
|
||||
Config is `[services.grafana]`, usually pointing at the shared `sherlock-cli`
|
||||
provider plus `base_url`. The first tool call authenticates lazily;
|
||||
`grafana-mcp --probe` verifies auth and `GET /api/user`.
|
||||
`sherlock logout grafana` clears the session.
|
||||
|
||||
## Why generic OAuth is not enough (and what is)
|
||||
|
||||
Grafana already federates with Authentik via
|
||||
`GF_AUTH_GENERIC_OAUTH_*`. That is **not** the same mechanism and does
|
||||
**not** make this MCP work:
|
||||
|
||||
| | `GF_AUTH_GENERIC_OAUTH_*` | `grafana-mcp` → API |
|
||||
|---|---|---|
|
||||
| OAuth client | **Grafana itself** | **sherlock** (`sherlock-cli` provider) |
|
||||
| Purpose | interactive browser login | programmatic API call |
|
||||
| Credential | a Grafana **session cookie** | an Authentik **JWT bearer** |
|
||||
| Grafana's job | requests the token, drops it for a cookie | **validate a token it never issued** |
|
||||
|
||||
Generic OAuth logs humans into the UI; it never accepts an externally
|
||||
minted bearer on `/api/*`. Presenting our bearer there makes Grafana
|
||||
treat it as a service-account token and return `Invalid API key`.
|
||||
|
||||
The mechanism that makes Grafana **accept** the Authentik JWT on its
|
||||
API is a separate integration: **`[auth.jwt]`**. It must be enabled on
|
||||
the Grafana deployment (it is missing from the `Charlie/victoriametrics`
|
||||
stack today). `auth.jwt` and `generic_oauth` coexist fine — UI users
|
||||
keep using SSO; sherlock uses JWT on the API.
|
||||
|
||||
## Required Grafana server config
|
||||
|
||||
Add to the `grafana` service environment in
|
||||
`Charlie/victoriametrics/docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
GF_AUTH_JWT_ENABLED: "true"
|
||||
GF_AUTH_JWT_HEADER_NAME: "Authorization" # Grafana strips the "Bearer " prefix
|
||||
GF_AUTH_JWT_USERNAME_CLAIM: "preferred_username" # matches the generic_oauth login attr → same user
|
||||
GF_AUTH_JWT_EMAIL_CLAIM: "email"
|
||||
GF_AUTH_JWT_JWK_SET_URL: "https://id.alexandru.macocian.me/application/o/sherlock-cli/jwks/"
|
||||
GF_AUTH_JWT_AUTO_SIGN_UP: "true"
|
||||
GF_AUTH_JWT_ROLE_ATTRIBUTE_PATH: "contains(groups, 'admins') && 'Admin' || 'Viewer'"
|
||||
GF_AUTH_JWT_ROLE_ATTRIBUTE_STRICT: "true"
|
||||
# All Authentik providers share one signing key, so pin the issuer
|
||||
# to sherlock-cli — otherwise any Authentik JWT would authenticate.
|
||||
GF_AUTH_JWT_EXPECT_CLAIMS: '{"iss":"https://id.alexandru.macocian.me/application/o/sherlock-cli/"}'
|
||||
```
|
||||
|
||||
The claim names above are verified against an actual `sherlock-cli`
|
||||
access token, which is a full RS256 JWT carrying: `iss`, `aud`,
|
||||
`preferred_username` (= the operator's username, same value as `sub`),
|
||||
`email`, `name`, and `groups` (including `admins`). Using
|
||||
`preferred_username` aligns with `GF_AUTH_GENERIC_OAUTH_LOGIN_ATTRIBUTE_PATH`,
|
||||
so JWT auth maps to the **same** Grafana account as browser SSO.
|
||||
|
||||
Grafana must be able to reach `GF_AUTH_JWT_JWK_SET_URL` from inside its
|
||||
container.
|
||||
|
||||
## Charlie defaults
|
||||
|
||||
| Variable / `-X main.…` | Charlie default |
|
||||
|------------------------|-----------------|
|
||||
| `grafanaIssuer` | `https://id.alexandru.macocian.me/application/o/sherlock-cli/` |
|
||||
| `grafanaBaseURL` | `https://grafana.alexandru.macocian.me` |
|
||||
| `grafanaClientID` | same Public `sherlock-cli` client as `gssh-mcp` |
|
||||
| `grafanaClientSecret` | `""` (Public, PKCE-only) |
|
||||
| `Version` | `0.0.0-dev` |
|
||||
|
||||
## Build & install
|
||||
|
||||
```bash
|
||||
cd ~/Dev/charlie/sherlock
|
||||
go install ./cmd/grafana-mcp
|
||||
```
|
||||
|
||||
That's it — the upstream tool set is compiled in. For other
|
||||
deployments:
|
||||
|
||||
```bash
|
||||
go build -tags noembed \
|
||||
-ldflags "-X main.grafanaIssuer=https://id.example/application/o/sherlock-cli/ \
|
||||
-X main.grafanaClientID=<CLIENT_ID> \
|
||||
-X main.grafanaBaseURL=https://grafana.example" \
|
||||
./cmd/grafana-mcp
|
||||
```
|
||||
|
||||
## Verify without an agent
|
||||
|
||||
```bash
|
||||
grafana-mcp --probe
|
||||
```
|
||||
|
||||
Expected once `[auth.jwt]` is configured:
|
||||
|
||||
```text
|
||||
OK: logged in to https://grafana.alexandru.macocian.me as <login> <<email>>
|
||||
```
|
||||
|
||||
A 401/403 (e.g. `Invalid API key`) means OAuth itself succeeded but
|
||||
Grafana is not yet validating the Authentik JWT for API access — apply
|
||||
the server config above. Force a re-login with `sherlock logout grafana`.
|
||||
|
||||
## Use with an agent
|
||||
|
||||
`sherlock copilot` / `sherlock claude` automatically render an MCP
|
||||
config listing `grafana-mcp` when the binary is installed. All exposed
|
||||
tools are read-only (subject to Grafana RBAC): dashboard
|
||||
search/inspection, datasource listing/querying (Prometheus, Loki, …),
|
||||
alert-rule and notification reads, annotations, and navigation
|
||||
deeplinks.
|
||||
|
||||
This relies on Grafana-side `[auth.jwt]` being configured (above). On a
|
||||
deployment where that isn't in place, the MCP still starts but every
|
||||
tool call 401s; verify with `grafana-mcp --probe` first.
|
||||
All exposed tools are read-only and still subject to Grafana RBAC.
|
||||
|
||||
+15
-55
@@ -1,61 +1,21 @@
|
||||
# gssh integration
|
||||
|
||||
How `gssh-mcp` will reuse the existing gssh server. Decided in Phase-0 planning; implementation in Phase 3.
|
||||
`gssh-mcp` is a client to the existing Gssh gateway. Sherlock does not open SSH
|
||||
connections, mint certificates, enforce host policy, or duplicate Gssh audit
|
||||
behavior.
|
||||
|
||||
## Why reuse
|
||||
Gssh owns JWT validation, host allow-lists, ephemeral SSH certificate handling,
|
||||
and command execution. Sherlock obtains the operator's OAuth token and passes it
|
||||
to `gssh-mcp`, which calls Gssh over HTTP and WebSocket.
|
||||
|
||||
The gssh server at `gssh.alexandru.macocian.me` already:
|
||||
- Authenticates inbound requests via Authentik JWTs (`HttpContext.User`).
|
||||
- Resolves the per-user host allow-list (`SessionController.GetHosts` → `SshConnectorService.Hosts`).
|
||||
- Establishes SSH sessions internally using the host's CA-signed credentials (the operator never sees an SSH key).
|
||||
- Streams stdin/stdout as a binary WebSocket and accepts text control messages for terminal resize (`SessionSocketRoute`).
|
||||
Current contract used by sherlock:
|
||||
|
||||
That's the entire job of "let an authorized human run a command on a Charlie host". Building a parallel SSH broker inside sherlock would duplicate the CA integration, the host-allow-list logic, and the audit trail — and put another piece of cert-handling code in our blast radius. We do not do that.
|
||||
| Endpoint | Purpose |
|
||||
| --------------------------------- | ----------------------------------------- |
|
||||
| `GET /api/v1/session/hosts` | Host allow-list. |
|
||||
| `POST /api/v1/session/initialize` | Ensure a Gssh session/certificate exists. |
|
||||
| `GET /api/v1/users/me` | Probe/debug identity. |
|
||||
| `WS /api/v1/exec/{host}` | Single-command execution stream. |
|
||||
|
||||
## What `gssh-mcp` actually is
|
||||
|
||||
A thin Go HTTP+WebSocket client to the existing gssh server, wrapped in a stdio MCP. It:
|
||||
|
||||
1. Reads `GSSH_TOKEN` from env at startup (an Authentik JWT for `aud=gssh`, written into the env by sherlock at agent spawn from the operator's stored TokenSet).
|
||||
2. Calls `POST /api/v1/session/initialize` with `Authorization: Bearer <jwt>` to materialise the user's session on the gssh server.
|
||||
3. Caches the list of permitted hosts from `GET /api/v1/session/hosts`.
|
||||
4. Exposes MCP tools:
|
||||
- `ssh.list_hosts()` — returns the cached host list.
|
||||
- `ssh.run(host, command, timeout?)` — opens a WebSocket to the session route for `host`, writes `command + "; echo __SHERLOCK_DONE__$?\n"`, reads until it sees the sentinel, returns `{stdout, exit_code}`.
|
||||
- `ssh.put_file(host, path, content)` (later) — same shell channel, base64-decode + `tee` on the far side, sentinel as above.
|
||||
5. On 401, re-reads the keyring (calling `authn.EnsureFresh`, which `flock`-serialises against concurrent MCP refreshes) and retries once.
|
||||
|
||||
No SSH key handling. No cert minting. No `~/Dev/gssh` shell-out.
|
||||
|
||||
## Endpoints sherlock relies on (gssh contract)
|
||||
|
||||
| Method | Path | Used for |
|
||||
|---|---|---|
|
||||
| `POST` | `/api/v1/session/initialize` | Materialise the user's session before opening the WebSocket. |
|
||||
| `GET` | `/api/v1/session/hosts` | Per-user host allow-list. |
|
||||
| `GET` | `/api/v1/users/me` | Sanity check / debug. |
|
||||
| `WS` | `/<session-socket-route>?hostName=<host>` (exact path defined in gssh's routing) | Bidirectional binary stream = stdin/stdout of the SSH session. Text frames are JSON control messages (e.g. `ResizeMessage`). |
|
||||
|
||||
If a gssh release ever changes one of these, `gssh-mcp` is the only thing in sherlock that needs to follow.
|
||||
|
||||
## Token shape
|
||||
|
||||
- Issuer: `https://id.alexandru.macocian.me/application/o/gssh/`
|
||||
- Audience: `gssh.alexandru.macocian.me`
|
||||
- Service kind in the registry: `oidc-federated`
|
||||
- `exchange.mode = "passthrough"` is safe because gssh already accepts the operator's Authentik ID token (same audience model gssh's web UI uses today). Switch to `rfc8693` only if we ever introduce per-tool scope splitting (e.g. read-only vs write).
|
||||
|
||||
## Completion sentinel — open detail for Phase 3
|
||||
|
||||
The gssh WebSocket is interactive (PTY-style). `gssh-mcp` needs a deterministic way to know a command has finished. Two options:
|
||||
|
||||
1. **Client-side wrap:** `gssh-mcp` sends `printf '%s\n' "<cmd>; echo __SHERLOCK_DONE__$?" | <session-shell>` and parses for the marker.
|
||||
2. **Server-side one-shot mode:** add a small endpoint / route flag on the gssh server that runs a single command and closes the socket with the exit code in the close frame. Cleaner, but it's a gssh change.
|
||||
|
||||
Phase 0 decision: try (1) first because it requires no gssh change; revisit (2) if we hit edge cases (TTY echo, prompt clutter, multi-line stdout truncation).
|
||||
|
||||
## Out of scope
|
||||
|
||||
- `gssh-mcp` does NOT speak gssh's WebSocket protocol "directly" by hand-rolling SSH packet framing. It just speaks the documented `/api/v1/session/*` HTTP + WS surface.
|
||||
- `gssh-mcp` does NOT shell out to `/mnt/seagate/Dev/gssh` (which is the C# server source, not a client binary).
|
||||
- `gssh-mcp` does NOT enforce per-host policy locally. The gssh server already does that against JWT claims; duplicating it client-side is a footgun.
|
||||
If that contract changes, `cmd/gssh-mcp/` is the only sherlock area that should
|
||||
follow.
|
||||
|
||||
+18
-147
@@ -1,157 +1,28 @@
|
||||
# gssh-mcp
|
||||
|
||||
sherlock's MCP server for **[Gssh](https://terminal.alexandru.macocian.me)**,
|
||||
the homelab SSH gateway. Lets an agent run a single shell command on
|
||||
an allow-listed host without standing up a PTY, using the operator's
|
||||
own JWT-authenticated session.
|
||||
Stdio MCP for the Gssh gateway. It lets an agent list allowed hosts, initialize
|
||||
the operator's Gssh session, and run one command on one host without exposing
|
||||
local SSH keys.
|
||||
|
||||
## How auth works
|
||||
Exact tool schemas live in `cmd/gssh-mcp/tools_*.go`.
|
||||
|
||||
Two Authentik OIDC providers front the same Gssh deployment:
|
||||
## Auth
|
||||
|
||||
1. **`gssh`** (Confidential) — the long-standing provider that powers
|
||||
Gssh's browser PTY login via server-side OIDC code flow.
|
||||
2. **`sherlock-cli`** (Public, PKCE) — a separate provider sherlock
|
||||
authenticates against from the CLI. No client secret, redirect URI
|
||||
pinned to `http://127.0.0.1:6990/callback`.
|
||||
`gssh-mcp` uses sherlock-managed OAuth with wallet key `gssh`, normally through
|
||||
the shared Authentik `sherlock-cli` provider. Gssh must trust that issuer and
|
||||
audience for bearer auth.
|
||||
|
||||
Gssh's JwtBearer scheme accepts JWTs from either provider via
|
||||
comma-separated `OIDC__ApiAudience` + `OIDC__BearerIssuers`. JWKS
|
||||
verification is anchored on a single discovery URL — all providers in
|
||||
an Authentik tenant share the signing key, so one fetch validates
|
||||
both.
|
||||
Config is `[services.gssh]` with provider or inline OAuth identity plus
|
||||
`base_url`. `sherlock logout gssh` clears the session.
|
||||
|
||||
The `sherlock-cli` provider can be reused by every future
|
||||
Authentik-fronted MCP: each backing service just adds the
|
||||
provider's audience to its own `ValidAudiences` and the provider's
|
||||
issuer to its own `ValidIssuers`. One wallet entry, one OAuth
|
||||
browser pop, N services.
|
||||
## Operation
|
||||
|
||||
## One-time setup
|
||||
When `gssh-mcp` is installed, `sherlock <agent>` includes it in the generated
|
||||
MCP config. The first tool call authenticates lazily. `gssh-mcp --probe`
|
||||
verifies auth against Gssh without an agent.
|
||||
|
||||
### Authentik
|
||||
`run_command` opens a WebSocket to Gssh's exec endpoint for the selected host.
|
||||
The server caps remote command runtime at 60 seconds; the client caps returned
|
||||
output at 256 KiB stdout and 64 KiB stderr.
|
||||
|
||||
Create a second OIDC provider:
|
||||
|
||||
1. **Applications → Providers → Create → OAuth2/OpenID Provider**:
|
||||
- **Client type:** Public
|
||||
- **Redirect URI (Strict):** `http://127.0.0.1:6990/callback`
|
||||
- **Subject mode:** `Based on the User's username` (the default
|
||||
"hashed user ID" mode produces a sub Gssh can't SSH as)
|
||||
- **Scope mappings:** include the standard openid/profile/email
|
||||
**and** a mapping that emits `groups` (Gssh's
|
||||
`OIDC__AllowedRoles` check reads it).
|
||||
2. **Applications → Applications → Create** an app that points at the
|
||||
new provider. Slug = `sherlock-cli` (the slug is the trailing path
|
||||
segment in the issuer URL).
|
||||
|
||||
### Gssh
|
||||
|
||||
Add the new audience and issuer to Gssh's env (alongside the existing
|
||||
gssh-provider values):
|
||||
|
||||
```env
|
||||
OIDC__ApiAudience=<gssh-client-id>,<sherlock-cli-client-id>
|
||||
OIDC__BearerIssuers=https://id.example/application/o/gssh/,https://id.example/application/o/sherlock-cli/
|
||||
```
|
||||
|
||||
## Build & install
|
||||
|
||||
For Charlie (default — Authentik client ID embedded in the binary):
|
||||
|
||||
```bash
|
||||
cd ~/Dev/charlie/sherlock
|
||||
go install ./cmd/gssh-mcp
|
||||
```
|
||||
|
||||
That's it. The Charlie Authentik `sherlock-cli` client ID lives in
|
||||
`cmd/gssh-mcp/gssh_clientid_charlie.go` and is baked in unless you
|
||||
build with `-tags noembed`.
|
||||
|
||||
For other deployments:
|
||||
|
||||
```bash
|
||||
go build -tags noembed \
|
||||
-ldflags "-X main.gsshIssuer=https://id.example/application/o/sherlock-cli/ \
|
||||
-X main.gsshClientID=<CLIENT_ID> \
|
||||
-X main.gsshBaseURL=https://gssh.example" \
|
||||
./cmd/gssh-mcp
|
||||
```
|
||||
|
||||
### Knobs
|
||||
|
||||
| Variable / `-X main.…` | Charlie default |
|
||||
|------------------------|-----------------------------------------------------------------------|
|
||||
| `gsshIssuer` | `https://id.alexandru.macocian.me/application/o/sherlock-cli/` |
|
||||
| `gsshBaseURL` | `https://terminal.alexandru.macocian.me` |
|
||||
| `gsshClientID` | embedded; see `gssh_clientid_charlie.go` |
|
||||
| `gsshClientSecret` | `""` (provider is Public, PKCE-only — no secret involved) |
|
||||
| `Version` | `0.0.0-dev` |
|
||||
|
||||
## Verify (without an agent)
|
||||
|
||||
```bash
|
||||
gssh-mcp --probe
|
||||
```
|
||||
|
||||
First run: opens a browser, you click through Authentik's "Authorize
|
||||
Sherlock-Cli" page, browser shows "Logged in. You may close this
|
||||
tab.", terminal prints:
|
||||
|
||||
```
|
||||
OK: logged in to https://terminal.alexandru.macocian.me as <name> (account=<username>, roles=[...])
|
||||
```
|
||||
|
||||
Subsequent runs are silent until the refresh window opens.
|
||||
|
||||
To force a re-login: `sherlock logout gssh`.
|
||||
|
||||
## Use with an agent
|
||||
|
||||
`sherlock copilot` (or `sherlock claude`) automatically renders an
|
||||
MCP config that lists `gssh-mcp`. Copilot spawns it as a stdio
|
||||
subprocess; the first call into a tool triggers the OAuth flow above.
|
||||
|
||||
### Tools
|
||||
|
||||
| Tool | Description |
|
||||
|-----------------------|----------------------------------------------------------------|
|
||||
| `list_hosts` | SSH hosts this operator is allowed to connect to. |
|
||||
| `initialize_session` | Force ephemeral 5-min SSH cert creation (idempotent). |
|
||||
| `run_command` | Run a single shell command on a host; returns stdout / stderr / exit code (or `timed_out` after the server's 60-second cap). |
|
||||
|
||||
`run_command` opens a fresh WebSocket per call to
|
||||
`/api/v1/exec/{host}`, sends a single `Start` datagram carrying the
|
||||
UTF-8 command, then demuxes streamed `Stdout` / `Stderr` datagrams
|
||||
into capped buffers until an `Exit` (carries ASCII exit code) or
|
||||
`Timeout` datagram arrives. Wire protocol lives in
|
||||
`internal/wsdatagram/` and mirrors Gssh's `Models/ShellDatagram.cs`
|
||||
exactly (fixed 2048-byte frame, 1-byte op + 2-byte length + 2045-byte
|
||||
payload).
|
||||
|
||||
### Buffer caps
|
||||
|
||||
| Stream | Cap | When exceeded |
|
||||
|---------|---------|------------------------------|
|
||||
| stdout | 256 KiB | trailing bytes dropped, `stdout_truncated=true` |
|
||||
| stderr | 64 KiB | trailing bytes dropped, `stderr_truncated=true` |
|
||||
|
||||
A `timed_out=true` result means the **remote command** exceeded the
|
||||
server-side 60-second hard cap. The client tool itself has a 90 s
|
||||
deadline so the agent isn't left hanging if the connection or the
|
||||
server stall mid-stream.
|
||||
|
||||
## Debugging
|
||||
|
||||
```bash
|
||||
SHERLOCK_DEBUG_LOG=/tmp/gssh-mcp.log gssh-mcp --probe
|
||||
tail -f /tmp/gssh-mcp.log
|
||||
```
|
||||
|
||||
Every HTTP request, WS dial, Start datagram, and terminal datagram
|
||||
gets one line. Empty `SHERLOCK_DEBUG_LOG` = no logging, no file
|
||||
touched.
|
||||
|
||||
When debugging Gssh-side rejections, look at Gssh's container logs —
|
||||
401s carry the JwtBearer failure reason (audience mismatch, issuer
|
||||
mismatch, expired, …) inline.
|
||||
Set `SHERLOCK_DEBUG_LOG` to capture MCP-side HTTP/WebSocket traces.
|
||||
|
||||
+24
-87
@@ -1,101 +1,38 @@
|
||||
# Installation
|
||||
|
||||
Sherlock installs from a clone of this repo with a single Go command.
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
git clone https://gitea.alexandru.macocian.me/amacocian/sherlock.git
|
||||
cd sherlock
|
||||
go run ./setup
|
||||
```
|
||||
|
||||
(`./install.sh` is kept as a thin convenience wrapper that just runs
|
||||
`go run ./setup` with the same flags.)
|
||||
|
||||
The installer:
|
||||
|
||||
1. **Syncs the source.** Clones (or updates) the sherlock repo under
|
||||
`${XDG_CACHE_HOME:-~/.cache}/sherlock/src` and checks out the latest
|
||||
release tag, so the install is reproducible and shares one checkout
|
||||
with `sherlock update`. (Use `--local` to build from the checkout
|
||||
you ran it from instead — handy for development.)
|
||||
2. **Seeds the config.** Copies [`config.example.toml`](../config.example.toml)
|
||||
to your config path (`~/.config/sherlock/config.toml` by default) — but
|
||||
only if you don't already have one, so re-runs never clobber filled-in
|
||||
values.
|
||||
3. **Opens it in your editor** (`$VISUAL` / `$EDITOR`, falling back to
|
||||
`nano`/`vi`) so you can fill in the Authentik issuer/client IDs and
|
||||
each service's base URL. See [configuration.md](configuration.md) for
|
||||
the schema.
|
||||
4. **Builds and installs** `sherlock` and every MCP binary
|
||||
(`gitea-mcp`, `grafana-mcp`, `gssh-mcp`) with `go install ./cmd/...`,
|
||||
baking in the release version.
|
||||
5. **Installs shell completions** where it can — currently a `fish`
|
||||
completion into `~/.config/fish/completions/sherlock.fish`, but only
|
||||
if you already have a fish config directory (it won't create one for
|
||||
non-fish users).
|
||||
|
||||
The exact same code runs on `sherlock update` — there is one installer,
|
||||
in `internal/installer`, with two entry points (`go run ./setup` for the
|
||||
bootstrap and `sherlock update` for self-update).
|
||||
|
||||
Then:
|
||||
|
||||
```bash
|
||||
sherlock copilot
|
||||
```
|
||||
Install from a clone with `go run ./setup`. `./install.sh` is a thin wrapper
|
||||
around the same command.
|
||||
|
||||
## Requirements
|
||||
|
||||
- The **Go toolchain** on `PATH` (the installer builds from source).
|
||||
- **git** on `PATH` (to clone the source; not needed with `--local`).
|
||||
- An **OS keyring** / Secret Service provider (the wallet lives there;
|
||||
see [storage.md](storage.md)).
|
||||
- The install dir (`go env GOBIN`, else `$(go env GOPATH)/bin`) on your
|
||||
`PATH` — this is standard Go setup. The installer notes the dir if it
|
||||
isn't already on `PATH`, but adding it is the usual one-time Go step
|
||||
(e.g. `fish_add_path -U ~/go/bin` on fish, or an `export PATH` line in
|
||||
`~/.bashrc`/`~/.zshrc`).
|
||||
- Go toolchain on `PATH`.
|
||||
- `git` on `PATH`, unless using `--local`.
|
||||
- OS keyring/Secret Service available.
|
||||
- The Go install directory on `PATH`.
|
||||
- The agent CLI you plan to launch (`copilot`, `claude`, ...).
|
||||
|
||||
## Flags
|
||||
## What setup does
|
||||
|
||||
Pass to `go run ./setup` (or `./install.sh`):
|
||||
The installer clones or updates a cached source checkout, checks out the latest
|
||||
release tag, seeds `config.toml` from `config.example.toml` without overwriting
|
||||
existing values, optionally opens the config in an editor, installs `sherlock`
|
||||
plus MCP binaries, and installs shell completions when supported.
|
||||
|
||||
| Flag | Effect |
|
||||
|---|---|
|
||||
| `-y`, `--yes` | Skip the editor step (config is already filled in). |
|
||||
| `--config-only` | Seed/edit the config but don't build or install. |
|
||||
| `--local` | Build from the current checkout instead of cloning to the cache (development). |
|
||||
| `-h`, `--help` | Show usage. |
|
||||
One implementation backs both bootstrap setup and `sherlock update`.
|
||||
|
||||
## Env overrides
|
||||
## Flags and environment
|
||||
|
||||
| Variable | Effect |
|
||||
|---|---|
|
||||
| `SHERLOCK_CONFIG` | Exact config path to write (else `$XDG_CONFIG_HOME` / `~/.config`). |
|
||||
| `SHERLOCK_UPDATE_REPO_URL` | Clone URL to build from (else the public sherlock repo). |
|
||||
| `VISUAL` / `EDITOR` | Editor to open. |
|
||||
Setup flags: `-y`/`--yes` skips the editor, `--config-only` edits config without
|
||||
installing, `--local` builds the current checkout, `-h`/`--help` prints usage.
|
||||
|
||||
Environment: `SHERLOCK_CONFIG` selects the config path,
|
||||
`SHERLOCK_UPDATE_REPO_URL` selects the source repo, and `VISUAL`/`EDITOR`
|
||||
selects the editor.
|
||||
|
||||
## Updating
|
||||
|
||||
Sherlock updates itself in place (running the same installer):
|
||||
Use `sherlock update` to install a newer release, or `sherlock update --force`
|
||||
to reinstall the latest release. Updates rebuild binaries and completions but do
|
||||
not touch existing config.
|
||||
|
||||
```bash
|
||||
sherlock update # if a newer release exists
|
||||
sherlock update --force # reinstall the latest regardless
|
||||
```
|
||||
|
||||
Release builds also print a one-line hint when a newer version is
|
||||
available. See [versioning.md](versioning.md).
|
||||
|
||||
## Re-running
|
||||
|
||||
`go run ./setup` (or `./install.sh`) is safe to re-run: it edits your
|
||||
existing config in place (never overwriting it) and re-installs the
|
||||
binaries. Use `-y` to skip the editor when you only want to rebuild, or
|
||||
`--config-only` to tweak the config without reinstalling.
|
||||
|
||||
To point sherlock at a different deployment, edit the values in your
|
||||
`config.toml` — there are no compiled-in defaults, so nothing is baked
|
||||
into the binaries. See [configuration.md](configuration.md).
|
||||
Re-running setup is safe: it preserves config and reinstalls binaries.
|
||||
|
||||
+21
-81
@@ -1,93 +1,33 @@
|
||||
# Storage
|
||||
|
||||
How sherlock persists secrets and runtime state.
|
||||
Sherlock stores credentials only in the OS keyring through `internal/keyring`.
|
||||
There are no plaintext token files and no sherlock daemon state.
|
||||
|
||||
## Decisions
|
||||
## Wallet
|
||||
|
||||
| Topic | Decision |
|
||||
|---|---|
|
||||
| Token storage | OS keyring via [`github.com/zalando/go-keyring`](https://github.com/zalando/go-keyring). No on-disk credential files, no `age` blobs, no plaintext. |
|
||||
| Wallet shape | One `TokenSet` per service (`gitea`, `grafana`, `miniflux`, …). Tracked via a sidecar `services-index` entry so `Store.List()` works on every backend without OS-specific search APIs. |
|
||||
| Pre-flight | The only constructor is `keyring.Open()`, which probes the keyring before returning a Store. There is no probe-less escape hatch. A missing/locked keyring returns `*UnavailableError` (with a populated `Hint` field) and exits 3. |
|
||||
| Keyring service ns | `sherlock` for all real entries; `sherlock-preflight` for the probe sentinel. Per-service tokens are accounts of the form `service:<name>`; the index is account `services-index`. |
|
||||
| Runtime files | `$XDG_RUNTIME_DIR/sherlock/<agent>.mcp.json` (0600), `$XDG_RUNTIME_DIR/sherlock.refresh.lock` (0600, flock anchor for cross-process refreshes). No socket, no PID file, no daemon log. |
|
||||
| Config files | `$XDG_CONFIG_HOME/sherlock/services.d/*.toml` (operator-registered services, Phase 2+). Agent integrations are compiled in — see [agents.md](agents.md). |
|
||||
Each service has one wallet entry under keyring service `sherlock` and account
|
||||
`service:<name>`. A `services-index` side entry makes `sherlock status` portable
|
||||
across keyring backends.
|
||||
|
||||
## TokenSet
|
||||
Stored data includes the service tokens, expiry times, OAuth
|
||||
issuer/client/scopes needed for refresh, and user identity metadata.
|
||||
|
||||
What sherlock stores per service entry:
|
||||
## Pre-flight
|
||||
|
||||
```go
|
||||
type TokenSet struct {
|
||||
IDToken string
|
||||
AccessToken string
|
||||
RefreshToken string
|
||||
IDExpiresAt time.Time
|
||||
RefreshExpAt time.Time
|
||||
Issuer string // for refresh: full OIDC issuer URL
|
||||
ClientID string // for refresh: OAuth client ID
|
||||
Scopes []string // for refresh: scopes to re-request
|
||||
Subject string
|
||||
Email string
|
||||
Name string
|
||||
}
|
||||
```
|
||||
`keyring.Open()` probes the OS keyring before returning a store. Missing or
|
||||
locked keyrings return `*keyring.UnavailableError` with a remediation hint;
|
||||
CLI/MCP callers exit with keyring failure rather than silently falling back.
|
||||
|
||||
Serialized as a single JSON blob per service entry — the keyring
|
||||
exposes one secret per `(service, account)` pair and we don't want to
|
||||
round-trip multiple secrets per session.
|
||||
## Runtime files
|
||||
|
||||
`Issuer` + `ClientID` + `Scopes` are persisted alongside the tokens so
|
||||
refresh and re-login work from any process, with no dependency on the
|
||||
shell environment that originally created the entry.
|
||||
Runtime files live under `$XDG_RUNTIME_DIR/sherlock/` or the sibling runtime
|
||||
directory:
|
||||
|
||||
## Wallet API
|
||||
- `<agent>.mcp.json`: generated MCP config, mode 0600.
|
||||
- `sherlock.refresh.lock`: cross-process token refresh lock.
|
||||
- `sherlock.login.lock`: cross-process first-login lock.
|
||||
|
||||
`keyring.Store` is service-keyed:
|
||||
## Platform note
|
||||
|
||||
```go
|
||||
type Store interface {
|
||||
Get(service string) (TokenSet, error) // ErrNoTokens if missing
|
||||
Set(service string, ts TokenSet) error // also updates the index
|
||||
Clear(service string) error // also updates the index
|
||||
List() ([]string, error) // names sorted
|
||||
}
|
||||
```
|
||||
|
||||
`Get` / `Set` / `Clear` go straight to the OS keyring under
|
||||
`(service="sherlock", account="service:<name>")`. `List` reads the
|
||||
sidecar `services-index` entry; `Set`/`Clear` keep it in sync.
|
||||
|
||||
## Pre-flight semantics
|
||||
|
||||
`keyring.Open()`:
|
||||
|
||||
1. Probes the keyring: writes a fixed sentinel value under service
|
||||
`sherlock-preflight` / account `probe`, reads it back, deletes it.
|
||||
2. On success: returns a live `Store` backed by the OS keyring.
|
||||
3. On failure: returns `*keyring.UnavailableError` whose `Cause` field
|
||||
wraps the underlying error and whose `Hint` field carries a
|
||||
per-OS one-line remediation. `Error()` includes both, so callers
|
||||
normally just print the error and move on; `keyring.IsUnavailable(err)`
|
||||
is the type-predicate for branching.
|
||||
|
||||
CLI behaviour: any failure prints the error (which already includes
|
||||
the hint) and exits with code `3`. MCPs inherit the same behaviour
|
||||
because they construct their Store via the same `keyring.Open()`.
|
||||
|
||||
## Platform notes
|
||||
|
||||
| OS | Backend | Common setup snag |
|
||||
|---|---|---|
|
||||
| Linux | Secret Service (D-Bus) | `gnome-keyring-daemon`, KWallet, or `keepassxc` with Secret Service enabled must be running for the session. Headless boxes need `gnome-keyring-daemon --components=secrets` started inside the session bus. |
|
||||
| macOS | Keychain | Works out of the box. First write may prompt for unlock. |
|
||||
| Windows | Credential Manager | Works out of the box. |
|
||||
|
||||
## Why not files
|
||||
|
||||
We considered an age-encrypted token blob and dropped it: the keyring
|
||||
gives us OS-managed locking, session affinity, and consistent
|
||||
multi-user behaviour for free, and avoids inventing a new key
|
||||
management story. The trade-off — Linux headless setups need a
|
||||
deliberate session keyring — is the right one for a homelab operator
|
||||
tool where the operator already has a desktop session.
|
||||
Linux needs a Secret Service provider available in the user session. macOS uses
|
||||
Keychain; Windows uses Credential Manager.
|
||||
|
||||
+16
-59
@@ -1,70 +1,27 @@
|
||||
# Versioning & self-update
|
||||
|
||||
Sherlock is versioned by git tags on the public repo and can update
|
||||
itself in place.
|
||||
`VERSION` at the repo root is the release source of truth.
|
||||
|
||||
## Version scheme
|
||||
## Releases
|
||||
|
||||
- The source of truth is the **`VERSION`** file at the repo root, holding
|
||||
a full semver string (e.g. `0.1.0`).
|
||||
- On every push to `main`, the [release workflow](../.gitea/workflows/release.yaml)
|
||||
runs the full CI (gofmt, vet, errcheck, staticcheck, `test -race`,
|
||||
build) and then — **only if every gate passed** — reads `VERSION` and,
|
||||
**if the matching tag `vVERSION` does not already exist**, creates and
|
||||
pushes it at that commit. A tag is never created on a red build.
|
||||
Cutting a release is a one-line edit to `VERSION` — no manual tagging.
|
||||
- Binaries bake their version in at build time via
|
||||
`-ldflags "-X main.Version=<v>"`. A build without that flag reports the
|
||||
sentinel `0.0.0-dev` and is treated as "not a release".
|
||||
On pushes to `main`, the release workflow runs formatting, static analysis, race
|
||||
tests, and build. If all gates pass, it creates tag `vVERSION` when that tag
|
||||
does not already exist.
|
||||
|
||||
## How a build gets its version
|
||||
Binaries receive their version at build time through linker flags. Builds
|
||||
without that flag report `0.0.0-dev` and skip passive update hints.
|
||||
|
||||
| Build path | Version baked |
|
||||
|---|---|
|
||||
| `go run ./setup` (default) | the latest `vX.Y.Z` tag in the cache checkout, or the `VERSION` file if there are no tags yet |
|
||||
| `go run ./setup --local` | the `VERSION` file in your working tree |
|
||||
| `sherlock update` | the latest published tag |
|
||||
| plain `go install ./cmd/...` | `0.0.0-dev` (no `-ldflags`) |
|
||||
## Update checks
|
||||
|
||||
## Update checking
|
||||
|
||||
`internal/selfupdate` reads the repo's tags through Gitea's public REST
|
||||
API (no auth) and compares the highest `vX.Y.Z` against the running
|
||||
binary using `golang.org/x/mod/semver`.
|
||||
|
||||
- **`sherlock version`** prints the running version and, for release
|
||||
builds, appends a one-line hint if a newer tag exists.
|
||||
- **Agent launches** (`sherlock copilot`, …) do a bounded, best-effort
|
||||
check just before handing off to the agent and print the same hint.
|
||||
It never blocks for more than ~1.5s and is silent on dev builds,
|
||||
offline, or when `SHERLOCK_NO_UPDATE_CHECK` is set.
|
||||
|
||||
The check is intentionally **uncached** — it runs each time — but is
|
||||
always bounded and non-fatal, so it can't break or noticeably delay a
|
||||
command.
|
||||
Release builds check the public Gitea tags API in two places: `sherlock version`
|
||||
and just before agent handoff. The check is bounded, best-effort, uncached, and
|
||||
disabled by `SHERLOCK_NO_UPDATE_CHECK`.
|
||||
|
||||
## Updating
|
||||
|
||||
```bash
|
||||
sherlock update # update to the latest release if newer
|
||||
sherlock update --force # reinstall the latest even if not newer
|
||||
```
|
||||
`sherlock update` installs the latest release through the shared installer.
|
||||
`--force` reinstalls even when already current. Updates rebuild binaries and
|
||||
completions but leave config untouched.
|
||||
|
||||
`update` resolves the latest tag, then calls the shared installer
|
||||
(`internal/installer`) — the same code `go run ./setup` runs. The
|
||||
installer clones (first time) or fetches the source under
|
||||
`${XDG_CACHE_HOME:-~/.cache}/sherlock/src`, checks out the latest tag,
|
||||
runs `go install -ldflags "-X main.Version=<tag>" ./cmd/...`, and
|
||||
installs shell completions. An update never touches your config.
|
||||
|
||||
There is no duplicate install logic: `go run ./setup` and
|
||||
`sherlock update` differ only in whether they seed/edit the config. Both
|
||||
require `git` and the Go toolchain on `PATH`.
|
||||
|
||||
## Environment overrides
|
||||
|
||||
| Variable | Effect |
|
||||
|---|---|
|
||||
| `SHERLOCK_NO_UPDATE_CHECK` | Disable the passive update hint. |
|
||||
| `SHERLOCK_UPDATE_TAGS_URL` | Override the tags API URL (forks, mirrors, tests). |
|
||||
| `SHERLOCK_UPDATE_REPO_URL` | Override the clone URL used by the installer. |
|
||||
Overrides: `SHERLOCK_UPDATE_TAGS_URL` for tag lookup and
|
||||
`SHERLOCK_UPDATE_REPO_URL` for clone/update source.
|
||||
|
||||
Reference in New Issue
Block a user