Skip to content

missing cost fixes - #137

Open
dixitaniket wants to merge 2 commits into
mainfrom
ani/missing-cost-fixes
Open

missing cost fixes#137
dixitaniket wants to merge 2 commits into
mainfrom
ani/missing-cost-fixes

Conversation

@dixitaniket

Copy link
Copy Markdown
Collaborator

No description provided.

@adambalogh
adambalogh marked this pull request as ready for review July 27, 2026 15:25
@adambalogh
adambalogh requested a review from Copilot July 28, 2026 14:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes cases where successful inferences were missing token-usage metadata (and therefore missing OpenGradient cost/settlement data), with a particular focus on xAI’s LangChain response shapes and streaming behavior.

Changes:

  • Expand extract_usage() to normalize usage from both LangChain’s usage_metadata and raw provider payloads under response_metadata (e.g., token_usage / usage).
  • Ensure xAI non-streaming requests don’t accidentally use a streaming-configured cached model, and ensure xAI streaming requests explicitly request usage when applicable.
  • Add targeted regression tests covering xAI streaming/non-streaming and the new extract_usage() fallbacks.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
tests/test_opengradient_field.py Adds regression tests ensuring xAI streaming/non-streaming paths propagate usage/cost behavior correctly.
tee_gateway/test/test_tee_core.py Adds unit tests for llm_backend.extract_usage() fallback behavior when LangChain omits usage_metadata.
tee_gateway/llm_backend.py Extends extract_usage() to support additional metadata shapes and normalize reasoning tokens.
tee_gateway/controllers/chat_controller.py Adjusts xAI streaming/non-streaming invocation kwargs and uses extract_usage() for streaming usage accumulation; logs when usage is missing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants