Skip Tool Normalization for GPT-OSS/Harmony Templates#1069
Open
sayanshaw24 wants to merge 5 commits into
Open
Conversation
added 5 commits
May 5, 2026 15:45
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the chat-templating pipeline to support GPT-OSS/Harmony templates that expect the original OpenAI tools shape ({ type: "function", function: {...} }) by conditionally skipping the existing tool “flattening” (NormalizeTools()) when the activated template references tool.function.
Changes:
- Add template-content detection in
ApplyChatTemplate()to decide whether to skip tool normalization for Harmony/GPT-OSS templates. - Add GPT-OSS/Harmony-focused unit tests covering basic rendering, tool definitions, tool-call flow, and regression coverage for non-Harmony templates.
- Add GPT-OSS test assets (
tokenizer_config.json,chat_template.jinja) to drive the new tests.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
shared/api/chat_template.cc |
Adds heuristic to bypass NormalizeTools() for templates that use tool.function. |
test/pp_api_test/test_tokenizer_chat.cc |
Adds GPT-OSS/Harmony template tests and a regression test ensuring Qwen normalization still applies. |
test/data/gpt-oss/tokenizer_config.json |
Introduces GPT-OSS tokenizer config used by the new tests. |
test/data/gpt-oss/chat_template.jinja |
Adds a Harmony-style chat template that accesses tool.function and renders tools in a TS namespace. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+423
to
+432
| // Look for "tool.function" but exclude "tool_call.function" matches | ||
| size_t pos = 0; | ||
| while ((pos = tmpl_str.find("tool.function", pos)) != std::string::npos) { | ||
| // Check that this isn't part of "tool_call.function" | ||
| if (pos < 5 || tmpl_str.substr(pos - 5, 5) != "call.") { | ||
| skip_tool_normalization = true; | ||
| break; | ||
| } | ||
| pos += 13; | ||
| } |
Comment on lines
+443
to
+445
| // GPT-OSS/Harmony: parse tools as-is without normalization | ||
| json tools_json = json::parse(message_obj["tools"].get<std::string>().c_str()); | ||
| message_obj["tools"] = tools_json; |
Comment on lines
+460
to
+461
| // GPT-OSS/Harmony: pass raw tools without normalization | ||
| tools_json = json::parse(tools_str.c_str()); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Skip tool normalization for GPT-OSS/Harmony chat templates
Summary
GPT-OSS (Harmony) models use chat templates that access
tool.functiondirectly, expecting the raw OpenAI tool format. The existingNormalizeTools()logic unwraps thefunctionobject from tools, which breaks these templates. This PR adds auto-detection to skip normalization when the template expects the original OpenAI structure.Problem
When tools are passed to
ApplyChatTemplate, they go throughNormalizeTools()which flattens the OpenAI{type: "function", function: {name, description, parameters}}format into a simpler structure. This works for Phi-4 and Qwen templates which expect flattened tools, but breaks GPT-OSS/Harmony templates which iterate overtool.function.name,tool.function.description, etc. directly.Solution
Added a template-content-based heuristic in
chat_template.cc:"tool.function"references"tool_call.function"(which is a different pattern)skip_tool_normalization = trueand pass tools as raw JSON without unwrappingThis is backward-compatible — existing Phi-4 and Qwen templates don't match the heuristic and continue to use
NormalizeTools()as before.Files Changed
shared/api/chat_template.ccNormalizeTools()for templates that accesstool.functiondirectlytest/pp_api_test/test_tokenizer_chat.ccTesting
test_tokenizer_chat.cccovering single tool, multi-tool, and tool result round-trip scenarios with a Harmony-style chat templategpt-oss-20b-generic-cpuin Foundry Local with the corresponding server-sideToolCallConfigchanges to be published in upcoming PR(s):tool_callsresponse