GetCachedModels to return valid and accurate entries

[Baiju Meswani (baijumeswani)]

This PR makes cached-model discovery resilient to catalog outages and returns every cached model variant instead of one selected model per alias.

• Falls back to foundry.modelinfo.json when all live catalog URLs fail.
• Treats valid disk models missing from metadata as BYOM models.
• Returns cached leaf variants in alias order and per-alias best-first order.

Issues addressed

1. Offline catalog returned no cached models

   • Before: if every live catalog request failed, catalog population returned an empty list even when models and foundry.modelinfo.json were present locally.
   • Now: when all live catalog URLs fail, the local snapshot supplies model metadata.

2. Locally valid models missing from metadata were omitted

   • Before: a downloaded model could be invisible when neither the live response nor snapshot contained its ID.
   • Now: valid models found by the disk scan are included with BYOM metadata.

3. Only one cached variant per alias was returned

   • Before: GetCachedModels() returned the alias container, which exposed only its selected CPU/GPU/NPU variant.
   • Now: it returns every cached leaf variant under the alias and skips only uncached variants.

4. Cache-only mode did not reconcile snapshot metadata with disk state

   • Before: snapshot entries were created without scanned local paths, and disk-only models were not represented.
   • Now: cache-only mode still performs no live requests, but attaches scanned paths and adds unmatched models as BYOM entries.

5. Multiple catalog URLs could produce duplicate model IDs

   • Before: each URL independently resolved local IDs and could synthesize duplicate entries.
   • Now: successful live results are aggregated, deduplicated by model ID, and merged with local models once.

6. Cached-model results could hide the canonical variant objects

   • Before: callers received alias containers rather than the leaf returned by GetModelVariant(model_id).
   • Now: both APIs return the same catalog-owned leaf pointer, keeping download, load, removal, and path state consistent.

Code flow

GetCachedModels()
       |
       v
Scan local model cache
       |
       +--> model IDs + local paths
       |
       v
Is catalog cache-only?
       |
   +---+---+
   |       |
  Yes      No
   |       |
   |       v
   |   Query configured live catalog URL(s)
   |       |
   |   +---+------------------+
   |   |                      |
   |   | Any URL succeeded    | All URLs failed
   |   v                      v
   | Aggregate + deduplicate  Load foundry.modelinfo.json
   | live metadata            (missing/invalid = empty)
   |   |                      |
   +---+----------------------+
       |
       v
Merge selected metadata with disk scan
       |
       +--> attach local paths by model ID
       +--> synthesize BYOM metadata for unmatched disk models
       |
       v
Save snapshot only when metadata came from live catalog
       |
       v
Group models by alias
       |
       v
Flatten and return every cached leaf variant

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
foundry-local	Ready	Preview	Aug 14, 2026 12:03am

[Copilot]

Pull request overview

Updates cached-model enumeration to scan disk directly without requiring catalog availability.

Changes:

• Enumerates all valid locally cached variants.
• Enriches entries from cached catalog metadata with disk-only fallback.
• Adds unit and cache-only integration coverage.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
sdk_v2/cpp/src/catalog/azure_model_catalog.cc	Implements disk-based cached-model enumeration.
sdk_v2/cpp/src/catalog/azure_model_catalog.h	Adds cached-model storage and synchronization.
sdk_v2/cpp/test/internal_api/local_model_scanner_test.cc	Tests variant enumeration and metadata fallback.
sdk_v2/cpp/test/sdk_api/cache_only_test.cc	Verifies cached models without catalog access.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (5)

sdk_v2/cpp/src/catalog/base_model_catalog.cc:174

• Calling GetCachedModels() after catalog population also resets all explicit SelectVariant() choices, even though cached-model enumeration should not mutate user selection. Re-sort to reflect refreshed metadata, but only update selection when the container is still using an automatically chosen default.

    for (auto& model : models_) {
      model->SortVariants();
      model->SelectDefaultVariant();
    }

sdk_v2/cpp/src/model.cc:445

• The early return for an already-cached model prevents a rescan from correcting a changed cache directory. This affects models without snapshot metadata, so GetCachedModels() can return a canonical model whose GetPath() still names the removed old directory. Compare and publish the newly scanned path instead of treating cached_ as proof that the path is current.

void Model::RefreshCachedPath(std::string local_path) {
  if (local_path.empty() || cached_.load(std::memory_order_acquire)) {
    return;

sdk_v2/cpp/src/catalog/base_model_catalog.cc:174

• A complete rescan never clears cache state for canonical IDs that disappeared from scanned_ids. If another process removes or invalidates a model directory, GetCachedModels() omits it, but previously returned pointers—and GetModelVariant() after population—still report IsCached() == true with the stale path. Reconcile absent previously-cached canonical leaves by publishing an empty path and clearing their cached flag.

  if (populated_) {
    for (auto& model : models_) {
      model->SortVariants();
      model->SelectDefaultVariant();
    }

sdk_v2/cpp/src/model.cc:440

• This only publishes the scanned path when the model was previously uncached. If the same model ID is rediscovered at a different directory after its old directory is moved or replaced, the canonical object returned by GetCachedModels() keeps the stale path. Publish an existing path whenever it differs from the current snapshot, even when cached_ is already true.

This issue also appears on line 443 of the same file.

  if (!local_path.empty() && !cached_.load(std::memory_order_acquire) && std::filesystem::exists(local_path)) {
    PublishLocalPath(std::move(local_path));
    cached_.store(true, std::memory_order_release);
  }

sdk_v2/cpp/src/catalog/base_model_catalog.cc:114

• This resets every existing container to its default variant on each catalog integration, including integrations that only refresh metadata. A variant explicitly chosen through the public SelectVariant() API is therefore silently replaced after a refresh. Preserve explicit selections while re-sorting; track whether selection is user-chosen and only recompute the default for new/default-selected containers.

This issue also appears on line 171 of the same file.

  for (auto& model : models_) {
    model->SortVariants();
    model->SelectDefaultVariant();

[Copilot]

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.

[Scott McKay (skottmckay)]

'leaf' isn't a concept a C++ API user knows anything about.

[Scott McKay (skottmckay)]

If catalog_result is from a snapshot do we need to call AddLocalModels?

I guess a new local model could have been added post-snapshot. Most likely it's just a bunch of de-duping to do from the call.

[Scott McKay (skottmckay)]

Can we do this at the same point we add the local model to model_infos?