feat(review-bot): stream Mistral, add an Anthropic backend and a TypeSafe model selector #1306

Merged

k.renner merged 2 commits from claude/review-bot-backends into master 2026-10-17 12:53:08 +00:00

Conversation 0 Commits 2 Files Changed 23

+1524 -211

claude-bot commented 2026-10-17 12:08:59 +00:00

Member

Copy Link

Copy Source

Why

Loki over the last 7 days: 53 reviews posted, 23 failed. About 30 failures/retries were mistral POST /conversations: HTTP 502, each arriving exactly 10:00 after the request started (retries spaced wait + 10 min). The bot sent one blocking, non-streaming request, so Mistral's edge cut every review that needed more than ten minutes, and every retry restarted the whole review until the 40-minute cap (context deadline exceeded).

What

• Mistral streams. stream: true (SSE: conversation.response.started with the conversation id, message.output.delta, tool.execution.started, conversation.response.done/.error). If the stream breaks off after the conversation started, the bot reads GET /conversations/{id}/history (up to 3 times, 30 s apart) and takes the answer once it ends in a verdict, before retrying. This needs store: true (it was false), so Mistral now keeps the conversations. A broken stream is retryable; a conversation.response.error event is not. Agents are now one per model, named pr-review-<model>; the old pr-review agent is left unused.
• Anthropic backend. Messages API through the Go SDK, streaming, adaptive thinking, effort: high. Anthropic connects to https://gitea-mcp.renner.uno/mcp itself. The toolset runs in allowlist mode (default_config.enabled=false), and only the read tools in MCP_TOOLS are enabled. Paused turns are resumed (max 30). Opus/Sonnet 5.5 use the server-side refusal fallback (fallbacks: default). SDK retries are off, so the bot's own retry policy (now also 529) is the only one.
• Models and selector. MODELS lists backend:model = description; the first is the default. With SELECTOR=typesafe, Jev (Choice) picks the model per PR from repo, title, description, size and changed files. A TypeSafe failure or an unusable choice falls back to the default. The model is logged with its probabilities and shown in the commit status. A backend whose keys are missing is switched off with a warning.
• Read-only by token. The MCP token is a second review-bot PAT scoped read:repository,read:issue,read:user, minted by a new init step rotate-mint-mcp into infrastructure/review-bot#mcp_token. The existing rotator policy already allows patching that path. REVIEW_TOKEN (write:repository) never goes to Anthropic.
• Secrets. All API keys come from the bot's own path apps/review-bot (minstral-api-key, antropic-api-key, typesafe-api-key, webhook-secret). The Mistral key moves off apps/wiki. Everything, including both tokens, is in review-bot-secrets.

Rollout

1. Done: bao kv patch homelab/infrastructure/review-bot mcp_token=placeholder. Without it, rotate-mint-mcp fails its field check and blocks the Woodpecker sync step.
2. After merge, run the rotator once (kubectl create job -n openbao --from=cronjob/review-bot-secret-rotate review-bot-secret-rotate-mcp) to mint the real MCP token. Until then MCP_TOKEN is the placeholder, and any review routed to Claude fails; the 30-minute sweep retries it. This also rotates REVIEW_TOKEN early (make-before-break).
3. The image builds through build-ci-images and rolls out through Flux image automation.

Tests

go vet, go test -race: fake SSE Mistral (deltas, chunk content, thinking dropped, broken stream recovered from history, broken stream retried, error event not retried), fake Anthropic stream (pause_turn resume, final text after the last tool call, MCP server and allowlist toolset in the request, effort/fallback/betas, 429 with Retry-After retryable, tool outside the list flagged), selector (chosen, failed → default, unknown → default), and config (backends without keys off, MODELS parsing).

Not verified live: Mistral's server-side behaviour on a dropped stream, and whether streaming avoids the 10-minute cut (if the edge caps total duration rather than idle time, the history recovery is the fallback).

🤖 Generated with Claude Code

## Why Loki over the last 7 days: 53 reviews posted, 23 failed. About 30 failures/retries were `mistral POST /conversations: HTTP 502`, each arriving **exactly 10:00 after the request started** (retries spaced wait + 10 min). The bot sent one blocking, non-streaming request, so Mistral's edge cut every review that needed more than ten minutes, and every retry restarted the whole review until the 40-minute cap (`context deadline exceeded`). ## What - **Mistral streams.** `stream: true` (SSE: `conversation.response.started` with the conversation id, `message.output.delta`, `tool.execution.started`, `conversation.response.done`/`.error`). If the stream breaks off after the conversation started, the bot reads `GET /conversations/{id}/history` (up to 3 times, 30 s apart) and takes the answer once it ends in a verdict, before retrying. This needs `store: true` (it was `false`), so Mistral now keeps the conversations. A broken stream is retryable; a `conversation.response.error` event is not. Agents are now one per model, named `pr-review-<model>`; the old `pr-review` agent is left unused. - **Anthropic backend.** Messages API through the Go SDK, streaming, adaptive thinking, `effort: high`. Anthropic connects to `https://gitea-mcp.renner.uno/mcp` itself. The toolset runs in allowlist mode (`default_config.enabled=false`), and only the read tools in `MCP_TOOLS` are enabled. Paused turns are resumed (max 30). Opus/Sonnet 5.5 use the server-side refusal fallback (`fallbacks: default`). SDK retries are off, so the bot's own retry policy (now also 529) is the only one. - **Models and selector.** `MODELS` lists `backend:model = description`; the first is the default. With `SELECTOR=typesafe`, Jev (Choice) picks the model per PR from repo, title, description, size and changed files. A TypeSafe failure or an unusable choice falls back to the default. The model is logged with its probabilities and shown in the commit status. A backend whose keys are missing is switched off with a warning. - **Read-only by token.** The MCP token is a second review-bot PAT scoped `read:repository,read:issue,read:user`, minted by a new init step `rotate-mint-mcp` into `infrastructure/review-bot#mcp_token`. The existing rotator policy already allows patching that path. REVIEW_TOKEN (write:repository) never goes to Anthropic. - **Secrets.** All API keys come from the bot's own path `apps/review-bot` (`minstral-api-key`, `antropic-api-key`, `typesafe-api-key`, `webhook-secret`). The Mistral key moves off `apps/wiki`. Everything, including both tokens, is in `review-bot-secrets`. ## Rollout 1. Done: `bao kv patch homelab/infrastructure/review-bot mcp_token=placeholder`. Without it, `rotate-mint-mcp` fails its field check and blocks the Woodpecker sync step. 2. After merge, run the rotator once (`kubectl create job -n openbao --from=cronjob/review-bot-secret-rotate review-bot-secret-rotate-mcp`) to mint the real MCP token. Until then MCP_TOKEN is the placeholder, and any review routed to Claude fails; the 30-minute sweep retries it. This also rotates REVIEW_TOKEN early (make-before-break). 3. The image builds through `build-ci-images` and rolls out through Flux image automation. ## Tests `go vet`, `go test -race`: fake SSE Mistral (deltas, chunk content, thinking dropped, broken stream recovered from history, broken stream retried, error event not retried), fake Anthropic stream (pause_turn resume, final text after the last tool call, MCP server and allowlist toolset in the request, effort/fallback/betas, 429 with Retry-After retryable, tool outside the list flagged), selector (chosen, failed → default, unknown → default), and config (backends without keys off, MODELS parsing). Not verified live: Mistral's server-side behaviour on a dropped stream, and whether streaming avoids the 10-minute cut (if the edge caps total duration rather than idle time, the history recovery is the fallback). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 1 commit 2026-10-17 12:08:59 +00:00

feat(review-bot): stream Mistral, add an Anthropic backend and a TypeSafe model selector ... 16d957dff1

Mistral cut every POST /conversations off with HTTP 502 exactly ten minutes
after it started (about 30 times in the last 7 days), and every retry restarted
the whole review, so a review needing more than ten minutes could never finish.

- Mistral: the conversation now streams (stream=true, SSE). A stream that
  breaks off after the conversation started is looked up in the stored history
  (store=true, GET /conversations/{id}/history) before it is asked again. A
  broken stream is retryable like 429/502/503/504; a
  conversation.response.error event is not. One agent per model, named
  <AGENT_NAME>-<model>.
- Anthropic: new backend on the Messages API (Go SDK, streaming, adaptive
  thinking, effort high). Anthropic connects to the Gitea MCP server
  (gitea-mcp.renner.uno/mcp) with a read-only token; the toolset is in
  allowlist mode with only the read tools enabled. Paused turns are resumed,
  refusals use the server-side fallback on Opus/Sonnet 5.5.
- Models: MODELS lists backend:model = description, the first is the default.
  A backend whose keys are missing is switched off with a warning instead of
  stopping the bot.
- Selector: SELECTOR=typesafe asks TypeSafe (Jev, Choice) which model reviews
  each PR, from the title, description, size and changed files. A failure or
  an unusable choice falls back to the default model.
- Secrets: the Anthropic and TypeSafe keys reuse the wiki keys
  (apps/ai/wiki-anthropic, apps/ai/wiki-typesafe), like the Mistral key does.
  The MCP token is a second review-bot PAT scoped read:repository,read:issue,
  read:user, minted by a new rotate-mint-mcp step into
  infrastructure/review-bot#mcp_token (seeded with a placeholder). Each
  optional key has its own ExternalSecret and optional env ref.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

claude-bot added 1 commit 2026-10-17 12:45:59 +00:00

fix(review-bot): read all API keys from apps/review-bot ... 7749636a8c

The Mistral, Anthropic and TypeSafe keys come from the bot's own OpenBao path
apps/review-bot (minstral-api-key, antropic-api-key, typesafe-api-key) instead
of the wiki's keys. All keys and both tokens are in review-bot-secrets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

k.renner merged commit 30a082864e into master 2026-10-17 12:53:08 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

apollo

Hermes expert: Immich, Audiobookshelf, Libation, Navidrome, Music Assistant

atlas

Hermes expert: Talos, Flux, network, Authentik, Matrix, monitoring

clio

Hermes expert: Paperless, wiki, ical-proxy, TRMNL, websites

daedalus

Hermes expert: Gitea, Woodpecker, Renovate, review-bot, Hermes

hestia

Hermes expert: Home Assistant, IoT, MQTT, ESPHome, voice

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

alert-monitor-bot claude-bot (Claude Bot) flux hermes-bot issue-fix-bot k.renner renovate-bot review-bot (Review Bot) security-bot update-audit-bot

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: konairius/homelab#1306