Skip to content

feat: add options.onlyLoaded to offer only loaded models - #49

Open
dgnsrekt wants to merge 2 commits into
agustif:mainfrom
dgnsrekt:feat/only-loaded
Open

feat: add options.onlyLoaded to offer only loaded models#49
dgnsrekt wants to merge 2 commits into
agustif:mainfrom
dgnsrekt:feat/only-loaded

Conversation

@dgnsrekt

Copy link
Copy Markdown

Closes #17.

Stacked on #48. That fix is a prerequisite — without it discovery returns nothing on current LM Studio builds, so this option cannot be exercised at all. Its commit is the first of the two here and will drop out of this diff once #48 merges. Review 1c40c7a alone for this feature.

Motivation

Discovery offers every generative record so LM Studio can load a model on demand when it is selected. That is the right default, but it has costs:

  • With a large library the picker fills with models that are merely downloaded. Prune old LLMs from the list #17 describes exactly this — a list that accumulates everything present "at some point in time."
  • On a shared or remote server, on-demand loading may be unwanted: picking an idle 400B model stalls the session while it loads, or evicts what someone else is using.

Change

provider.lmstudio.options.onlyLoaded restricts generated models and the generated whitelist to records with a non-empty loaded_instances:

{
  "provider": {
    "lmstudio": {
      "options": { "onlyLoaded": true }
    }
  }
}

Follows the options.* convention used for provider-scoped plugin settings (same shape as options.badge in #43).

Default behavior is unchanged — the option is opt-in, and only the boolean true enables it, so a malformed value can't silently empty the model list. When nothing is loaded the plugin generates an empty model list rather than falling back to idle entries; the discovery log gains onlyLoaded and skippedUnloaded so the filter is visible.

One incidental benefit: when a key has both a loaded and an unloaded record, the filter keeps the loaded one, so limit.context reflects the live allocation rather than whichever record landed last.

Verification

9 tests added (28 → 37): opt-in filtering, whitelist narrowing, default-unchanged, empty-when-nothing-loaded, explicit-override precedence, and a fail-open matrix over undefined | false | "true" | 1 | null.

npm run validate clean (lint, typecheck, 37 tests, build).

End-to-end against LM Studio 0.4.x, 37 records / 4 loaded instances, via opencode models lmstudio:

config models offered
onlyLoaded absent 29
onlyLoaded: true 2

The 2 match the MODEL column of lms ps exactly (google/gemma-4-26b-a4b-qat, qwen/qwen3.8-27b).

Observation, not addressed here

That run also surfaced something pre-existing: my server reports the same key from more than one host, so loaded_instances for one key spans several machines and three native records collapse into two model entries via Object.fromEntries. It's why discovered counts records while the map holds fewer. I left the semantics exactly as they are on main rather than widen this PR — it looks related to #42, and I'm happy to open a separate issue with the payload if useful.

🤖 Generated with Claude Code

https://claude.ai/code/session_01S7Wjrr4wy4Eiig1PqtHfpD

run2dos and others added 2 commits August 14, 2026 17:53
`capabilities.reasoning` pinned `allowed_options` and `default` to
`off|on|low|medium|high`. LM Studio now publishes an `xhigh` level, and
because the enum sits inside the response schema a single model carrying
it fails `LMStudioModelsResponseSchema.safeParse` for the entire payload.
`discoverModels` then throws `unsupported response`, so discovery yields
no models at all and the provider falls back to models.dev entries the
user has not downloaded.

The reasoning block is parsed but never mapped: the v1 contract leaves
OpenCode's `reasoning` flag unset until both projects publish an
interoperable mapping. Validating these values strictly gains nothing and
costs total discovery whenever LM Studio adds a level.

Accept any string and keep the known levels as an exported constant for
documentation, so a future level cannot break discovery the same way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S7Wjrr4wy4Eiig1PqtHfpD
Discovery offers every generative record so LM Studio can load a model on
demand when it is selected. With a large library most entries are idle,
and the picker fills with models that are merely downloaded (agustif#17). On
shared or remote servers, on-demand loading may not be wanted at all.

`provider.lmstudio.options.onlyLoaded` restricts the generated models and
whitelist to records with a non-empty `loaded_instances`, so the list
matches `lms ps`. Default behavior is unchanged: the option is opt-in and
only the boolean `true` enables it, so an unusable value cannot silently
empty the model list.

When nothing is loaded the plugin generates an empty model list rather
than falling back to idle entries, and the discovery log reports
`onlyLoaded` and `skippedUnloaded` so the filter is visible.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S7Wjrr4wy4Eiig1PqtHfpD
@dgnsrekt

Copy link
Copy Markdown
Author

Filed the duplicate-key observation from the PR description as #50, with a reproduction and the sanitized record shapes. It reproduces on main and is independent of this PR, so nothing here changes.

One thing that fell out of writing it up: onlyLoaded is immune to that collapse. The filter runs before the map is built, so when a key has both a loaded and an idle record the idle one is removed first and the loaded record survives — verified, not just reasoned about. That is the incidental benefit mentioned above, and it means this PR is unaffected either way #50 is resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prune old LLMs from the list

2 participants