Skip to content

feat: add NVIDIA provider as supported model provider#3421

Open
Priyanshu-sde wants to merge 2 commits into
docker:mainfrom
Priyanshu-sde:main
Open

feat: add NVIDIA provider as supported model provider#3421
Priyanshu-sde wants to merge 2 commits into
docker:mainfrom
Priyanshu-sde:main

Conversation

@Priyanshu-sde

Copy link
Copy Markdown

Summary

Add first-class support for NVIDIA (NIM / build.nvidia.com) as an OpenAI-compatible model provider. Hosted NVIDIA models work via a new alias, while self-hosted NVIDIA NIM is supported by overriding base_url.

Changes

  • Added a nvidia alias in pkg/model/provider/aliases.go:
    • APIType: "openai"
    • BaseURL: "https://integrate.api.nvidia.com/v1"
    • TokenEnvVar: "NVIDIA_API_KEY"
  • Added nvidia to TestCatalogAliases in pkg/model/provider/aliases_test.go.
  • Added examples/nvidia.yaml demonstrating NVIDIA NIM with the nvidia/nemotron-3-super-120b-a12b model.

Notes

  • The example model exists in pkg/modelsdev/snapshot.json, allowing example validation without special-casing.
  • Self-hosted NVIDIA NIM is supported by overriding base_url to point to a local NIM endpoint.

Testing

  • task test — all tests pass.
  • task lint — no issues.

@Priyanshu-sde Priyanshu-sde requested a review from a team as a code owner July 2, 2026 15:32
@Priyanshu-sde

Copy link
Copy Markdown
Author

This pr closes #3356

@Sayt-0 Sayt-0 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding NVIDIA NIM as a built-in alias. The core wiring is correct (APIType, base_url, TokenEnvVar), TestCatalogAliases covers it, and the example model resolves against the models.dev snapshot, so TestParseExamples passes without special-casing. A few items should be addressed before this is a complete first-class provider, matching how the other OpenAI-compatible aliases are wired.

Functional

  • Register nvidia in openModelHostProviders (pkg/model/provider/openai/client.go). NVIDIA NIM fronts open-weight models (Nemotron, Llama, Qwen, DeepSeek) whose chat templates often reject more than one system message. Without this entry, the per-source system messages are not coalesced, which reproduces the empty-stream / System message must be at the beginning failures (#3145, #2327, #3344). The shipped example already emits multiple system messages (agent instruction plus the filesystem, shell, and think toolsets), so it is likely to hit this path.
  • Add a row for nvidia to the openAIAliasProviders end-to-end table (pkg/model/provider/openai_alias_providers_test.go) with mergesSystemMessages: true, matching every other alias.

Docs (each existing alias ships these)

File Content
docs/providers/overview/index.md built-in providers table
docs/concepts/models/index.md providers table
docs/configuration/models/index.md provider enumerations (lines 20, 51, 403)
docs/providers/nvidia/index.md dedicated provider page (missing)

Optional (non-blocking)

  • agent-schema.json provider description ends in etc., so adding nvidia there is nice-to-have.
  • Absence from cloudProviders / DefaultModels in pkg/config/auto.go is consistent with nebius / xai / minimax (nvidia simply will not participate in auto-selection). NVIDIA_API_KEY is still forwarded as a credential via ProviderAPIKeyEnvVars() since it iterates the aliases.

See inline comments for the specific anchors.

Comment thread pkg/model/provider/aliases.go
Comment thread pkg/model/provider/aliases_test.go
Comment thread examples/nvidia.yaml
@aheritier aheritier added area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Jul 2, 2026
@Priyanshu-sde

Copy link
Copy Markdown
Author

Thanks so much for the detailed and thorough review, @Sayt-0 ,
I really appreciate you taking the time to point out the exact anchors instead of just flagging the gaps. It made it easy to understand it better.
I've pushed the fixes:

  • Registered nvidia in openModelHostProviders so consecutive system messages get coalesced.
  • Added a nvidia row to openAIAliasProviders with mergesSystemMessages: true.
  • Added the # yaml-language-server: $schema=../agent-schema.json header to examples/nvidia.yaml.

This is genuinely one of the more educational reviews I've gotten. I'll keep digging into the codebase and try to pick up more issues I can meaningfully contribute to. Thanks again for the guidance!

@Priyanshu-sde Priyanshu-sde requested a review from Sayt-0 July 3, 2026 12:11
@Sayt-0

Sayt-0 commented Jul 3, 2026

Copy link
Copy Markdown
Member

Hi @Priyanshu-sde ! You're welcome. Don't hesitate if you want to contribute again !

@Sayt-0

Sayt-0 commented Jul 3, 2026

Copy link
Copy Markdown
Member

The changes looks good. Approved ! Have a nice day !

@Sayt-0

Sayt-0 commented Jul 3, 2026

Copy link
Copy Markdown
Member

one last thing: your commits must be signed in this repo

@Priyanshu-sde

Copy link
Copy Markdown
Author

Hi @Sayt-0 , I have updated to signed commits, also that's a new thing i got to learn about Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants