test: port the cc_convert conversion fixture corpus#143
Merged
Conversation
…transforms Vendor the golden conversion fixtures from cc_convert (github.com/yitianlian/cc_convert, MIT OR Apache-2.0; redistributed under MIT with attribution README + license text) and run the pure Python transforms in agentix.bridge.clients._anthropic_transforms against them: - requests (32 cases): anthropic_messages_to_openai vs golden, with a documented normalizer (null==absent, text-only content-list folding, empty tool description == absent). - responses (6 cases): openai_to_anthropic_messages vs golden, with only the generated message id excluded. - streams (5 cases): no incremental OpenAI-SSE seam exists in the Python path, so the tests assert final-assembled-response equivalence through openai_to_anthropic_messages + anthropic_sse; one case is skipped because its golden is itself a LiteLLM stream-translator artifact. Every known Python-path gap is xfail(strict=True) with a precise reason, so the xfail table is the prioritized gap list: images, tool_choice (any/named/auto/none), thinking/reasoning_effort, thinking history, stop_sequences, top_k/stream passthrough, metadata.user_id, >64-char tool names, cached-token usage, empty content handling, aborted-stream stop_reason, reasoning deltas. Result: 21 passed, 1 skipped, 22 xfailed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports the conversion fixture corpus from the deferred #122 stage F tree (fixtures only — zero Rust, zero scripts) and runs the Python Anthropic↔OpenAI transforms (
agentix.bridge.clients._anthropic_transforms) against it as a parametrized golden-parity suite.plugins/abridge/tests/fixtures/anthropic_openai/: 32 request cases (anthropic/openai/tool_map triples), 6 response cases, 5 stream cases.xfail(strict=True)with a precise reason — the xfail table doubles as the prioritized gap list (22 entries): image blocks,tool_choiceany/named/auto/none, thinking/reasoning (requests and streams),stop_sequences,metadata.user_id,top_k, >64-char tool names, cached-token usage, empty-content shape, stream-without-finish_reason.anthropic_sse), so the suite tests final-assembled-response equivalence and says so; chunk-fidelity is out of scope until a streaming seam exists.Notable findings
max_completion_tokensgap does not manifest — the golden keepsmax_tokensfor o3-mini and the case passes.29_two_parallel_tool_calls: LiteLLM artifact merges two parallel tool calls into one block with concatenatedpartial_json); skipped with the reason recorded, not xfailed, so the xfail list stays a pure gap list.Testing
test_anthropic_transform.py: 9 passed, unchangedruff checkcleanCloses #141.
🤖 Generated with Claude Code