Skip to content

feat(cli): guide users from "no API key" to a working model (actionable errors, doctor, setup, tutorial) #3442

Description

@Sayt-0

Summary

First contact with docker-agent on a machine without provider credentials is confusing.
Typing just docker agent can silently start a TUI on a model that is not pulled and fail
with a raw HTTP 404 at the first message, or exit 0 with no output at all in a non-TTY
context. Error messages mention concepts that are never named ("built-in environment
variable providers"), --env-from-file failures are silently swallowed, and no command
answers "what is my model/credential state?". Docs cover each topic in isolation but no
tutorial connects run-time failures to fixes.

Current behavior (reproduced on 1138afa)

Bare docker agent (no args, implicit run of the default agent, model auto:
anthropic -> openai -> google -> mistral -> ... -> dmr/ai/qwen3):

# Environment Result Problem
1 TTY, no keys, DMR not installed AutoModelFallbackError: pull / install / configure one of ~20 env vars Correct but a wall of text; no interactive next step, no docker agent models or docs link
2 TTY, no keys, DMR endpoint reachable but docker model CLI plugin missing/broken TUI starts silently on dmr/ai/qwen3 (not pulled); first message fails with raw HTTP 404: POST .../chat/completions "Not installed" detection is an exact string match on "unknown flag: --json" (dmr/client.go:88-94); any other status-query failure is logged and the pull skipped. The DMR HTTP API had already reported 0 models before the message was sent
3 non-TTY (pipe, CI) Exit code 0, zero output Silent no-op; no "needs a terminal, use --exec" error (getting-started does check isatty)

Other scenarios:

# Scenario Result Problem
4 run agent.yaml (cloud model), no keys RequiredEnvError: "...Store those secrets using one of the built-in environment variable providers." Providers never named, no example command, no docs link, no local-model alternative
5 run --exec hitting the scenario-2 state ❌ model failed: ... 404 ... Error: model failed: ... 404 Error printed twice (CLI printer + cobra)
6 run --env-from-file typo.env Same "must be set" error as #4 Missing/malformed file only logged via slog (invisible without --debug), run continues; the flag recommended by error #4 itself no-ops on a typo
7 "Am I set up?" debug auth shows only the Docker Desktop JWT Nothing reports providers with credentials (and their source), DMR reachability, pulled models, or what auto would pick (see #2799)
8 Quickstart Option A "just start chatting", Option D uses anthropic/... No mention that a key or a pulled local model is required first

Proposed behavior

Flow to reach: every failure names the exact next command.

docker agent            (bare, TTY)
  |- credentials or pulled DMR model found -> run as today
  |- nothing found -> offer `docker agent setup` (pick provider+key, or DMR+pull)
docker agent            (bare, non-TTY)
  `- error: "interactive TUI needs a terminal; use --exec" (exit != 0)
docker agent run ...    (missing credentials, non-TTY or declined setup)
  `- error listing concrete fixes + docs URL
docker agent doctor
  `- full credential/model state table at any time

Phase 1: actionable errors (S)

  • Bare docker agent in a non-TTY context errors out like getting-started instead of
    exiting 0 silently.
  • DMR: replace the exact-string "not installed" detection; on any status-query failure,
    verify model availability through the DMR HTTP API (already queried for listing) and
    fail at client creation with "model X is not available in Docker Model Runner; run
    docker model pull X" instead of a raw HTTP 404 at message time.
  • RequiredEnvError (and the first_available variant) names each secret source with a
    one-line example (export, --env-from-file, keychain, pass, Docker Desktop,
    credential helper), adds a docs URL, and mentions the local-model alternative
    (--model dmr/..., docker agent models).
  • --env-from-file: unreadable or unparseable file aborts the run with a clear error
    instead of slog + continue.
  • Exec mode prints the final error once, not twice (CLI printer + cobra both print today).

Phase 2: docker agent doctor (M)

  • New command (or debug env) printing: provider | credential found | source
    (env / env-file / compose secret / helper / Desktop / pass / keychain), DMR
    installed/reachable + pulled models, and the model auto would select.
  • Optional docker agent doctor agent.yaml: check the file's specific requirements
    (models + toolset env like GITHUB_PERSONAL_ACCESS_TOKEN).
  • Values always redacted; --json for scripting.

Phase 3: guided setup (M)

  • docker agent setup: interactive wizard. Cloud path: pick provider, paste key,
    choose where to store it (env file, keychain, pass). Local path: check DMR,
    pick and pull a model. Ends with a ready-to-copy docker agent run line.
  • Offered automatically when bare docker agent or an interactive run finds no usable
    model (TTY only, decline-able), complementing the first-run tour from feat: interactive onboarding tour on first run, replayable on demand #3426.

Phase 4: docs tutorial (S)

  • New getting-started page "Set up a model (API key or local)" walking both paths
    end to end, linked from README, quickstart Options A/D, and the Phase 1 messages.

Acceptance criteria

  • Bare docker agent never starts a session that cannot answer: either a usable model
    exists, or the user gets setup guidance (TTY) / an actionable error (non-TTY, exit != 0).
  • A DMR model that is not pulled is reported as such at startup, never as an HTTP 404
    at message time, whether or not the docker model CLI plugin works.
  • Every credential error names the exact commands to fix it and links to docs.
  • --env-from-file with an unreadable or malformed file fails the run explicitly.
  • docker agent doctor reports credential sources, DMR state, and auto-selection result.

Implementation notes (starting points)

  • Bare-command dispatch: cmd/root/root.go:121-128 (RunE -> run); isatty precedent in
    cmd/root/getting_started.go
  • pkg/model/provider/dmr/client.go:~85-101 (exact-string ErrNotInstalled match; status
    failure skips pull); models listing over HTTP already exists (Listed DMR models);
    pull UX in pkg/model/provider/dmr/pull.go
  • pkg/environment/errors.go (RequiredEnvError.Error), same advice text duplicated in
    pkg/config/first_available.go:~180; raised from pkg/config/config.go:157
  • pkg/config/runtime.go:143-161 (computedEnvProvider swallows env-file errors);
    early validation could live at flag handling in cmd/root/flags.go:30
  • pkg/config/auto.go (AutoModelFallbackError, AvailableProviders, cloudProviders
    table reusable by doctor/setup); cmd/root/models.go for credential-aware listing
  • cmd/root/debug_auth.go as scaffold for doctor; secret sources in pkg/environment/
    (default.go, keychain.go, pass.go, docker-desktop.go, credential_helper.go)
  • Double print: pkg/cli/printer.go:67 + cobra error printing on the run path
  • Docs: docs/getting-started/quickstart/index.md, docs/_data/nav.yml

Open questions

  • Command naming: doctor vs debug env vs extending models --verbose.
  • Where setup stores keys by default (per-OS: keychain on macOS, pass on Linux,
    fallback ~/.config/cagent/.env?).
  • Whether setup should also be offered on first run before the tour, or only on failure.

Phases are independently shippable; Phase 1 alone removes most of the confusion.

Metadata

Metadata

Assignees

Labels

area/cliCLI commands, flags, output formattingarea/configFor configuration parsing, YAML, environment variablesarea/docsDocumentation changesarea/modelsLLM model integrations and model providersarea/providersFor features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.)area/providers/docker-model-runnerDocker Model Runner (DMR) local inferencekind/featPR adds a new feature (maps to feat:). Use on PRs only.

Fields

No fields configured for Enhancement.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions