fix(ask): judge every question in one request #10

2026-09-25T06:00:44Z

claude-bot commented

2026-09-25 06:00:44 +00:00

The ask use case (ported from wiki_ask.py) sent each context choice to TypeSafe in its own request and batched the rest, on the assumption that a 254-option choice dominates latency. TypeSafe's docs say the opposite: every question is scored in isolation against the same state in one go, adding questions barely changes response time, and their parallel-questions cookbook measured one batched call as 12x cheaper and 10x faster than per-question calls because the state is what costs.

Live before: one question → 2 requests, 22,957 input tokens, 1.11 s. After deploy I will re-run the same question and post the numbers here.

JudgeInParallelAsync → JudgeAsync: one JudgeRequest with all expanded questions.
AskOptions.MaxParallel removed (nothing read it outside the use case).
Test asserts one request carrying all three questions and Usage.Requests == 1.
Full suite: 128 green.

The vault's Python fallback (bin/wiki_ask.py) gets the same change in its own commit.

🤖 Generated with Claude Code

The ask use case (ported from `wiki_ask.py`) sent each context choice to TypeSafe in its own request and batched the rest, on the assumption that a 254-option choice dominates latency. TypeSafe's docs say the opposite: every question is scored in isolation against the same state in one go, adding questions barely changes response time, and their parallel-questions cookbook measured one batched call as 12x cheaper and 10x faster than per-question calls because the state is what costs. Live before: one question → 2 requests, 22,957 input tokens, 1.11 s. After deploy I will re-run the same question and post the numbers here. - `JudgeInParallelAsync` → `JudgeAsync`: one `JudgeRequest` with all expanded questions. - `AskOptions.MaxParallel` removed (nothing read it outside the use case). - Test asserts one request carrying all three questions and `Usage.Requests == 1`. - Full suite: 128 green. The vault's Python fallback (`bin/wiki_ask.py`) gets the same change in its own commit. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 1 commit 2026-09-25 06:00:44 +00:00

fix(ask): judge every question in one request

ci/woodpecker/pr/test Pipeline was successful

Details

ci/woodpecker/push/test Pipeline was successful

Details

347ac9d7a6

The state dominates a System One request and each question is scored in
isolation against it, so sending context choices separately only re-sent
the state per request and added round trips. A live ask carried 22,957
input tokens over two requests for one question; one request carries it
once. Drops the now unused AskOptions.MaxParallel.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

k.renner merged commit 4cd8ef1dd4 into main

2026-09-25 06:01:55 +00:00

k.renner deleted branch fix/judge-one-request

2026-09-25 06:01:56 +00:00

k.renner referenced this issue from a commit

2026-09-25 06:01:56 +00:00

Merge pull request 'fix(ask): judge every question in one request' (#10) from fix/judge-one-request into main

claude-bot commented

2026-09-25 06:08:50 +00:00

Measured on build.1790316170, same question ("Does the UDM intercept DNS queries from the LAN?", k=6), through wiki-index.renner.uno:

	before (build.1790261926)	after
judge requests	2	1
judge input tokens	22,957	12,253
judge elapsed	1.11 s	0.94 s first run, 0.30 s on the next two
covered	0.97	0.97–0.98
evidence to the answer model	udm-transparent-dns-interception#p1 (p=1.00)	same

Tokens −47 %, one round trip fewer, no change in what the judge or the answer model sees. The vendor's latency claim holds for jev with a 254-option choice in the batch.

Measured on build.1790316170, same question ("Does the UDM intercept DNS queries from the LAN?", k=6), through wiki-index.renner.uno: | | before (build.1790261926) | after | |---|---|---| | judge requests | 2 | 1 | | judge input tokens | 22,957 | 12,253 | | judge elapsed | 1.11 s | 0.94 s first run, 0.30 s on the next two | | covered | 0.97 | 0.97–0.98 | | evidence to the answer model | udm-transparent-dns-interception#p1 (p=1.00) | same | Tokens −47 %, one round trip fewer, no change in what the judge or the answer model sees. The vendor's latency claim holds for jev with a 254-option choice in the batch.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: konairius/Wiki#10