Skip to content

Fail fast on rejected keys and unknown models - #381

Merged
rejojer merged 1 commit into
mainfrom
staging/flash
Aug 2, 2026
Merged

Fail fast on rejected keys and unknown models#381
rejojer merged 1 commit into
mainfrom
staging/flash

Conversation

@rejojer

@rejojer rejojer commented Aug 2, 2026

Copy link
Copy Markdown
Member

llm_completion and llm_acompletion caught every exception the same way:
retry ten times a second apart, then return "". A rejected key or a model
name that does not exist fails identically on all ten, so a run with a bad key
took half a minute to produce a document of empty summaries and exit 0.

They now re-raise on 401, 403 and 404. Those are misconfiguration: every later
call fails the same way, and no caller can do anything useful with the empty
string. Everything else keeps the existing retry, including an unknown status,
which is a transport failure.

400 is deliberately not in the set. It also carries context_length_exceeded,
which is a per-prompt failure on one oversized node rather than a broken setup,
and callers absorb that today. Promoting it would turn one empty summary into a
failed run.

tree_optimize.expand caught the new exception and logged it per node, so the
same misconfiguration went quiet on the --optimize path. It re-raises too.

Measured with an invalid key: a default run now stops in 7s and --optimize in
2.4s, both naming the 401, where before both finished with empty summaries.

@rejojer
rejojer merged commit fb3f6e4 into main Aug 2, 2026
4 checks passed
@rejojer
rejojer deleted the staging/flash branch August 2, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant