Skip to content

feat(conformance): vendor API contract; add staleness, type, and baseline checks#47

Merged
vdavez merged 1 commit into
mainfrom
feat/contract-first-conformance
Jun 10, 2026
Merged

feat(conformance): vendor API contract; add staleness, type, and baseline checks#47
vdavez merged 1 commit into
mainfrom
feat/contract-first-conformance

Conversation

@makegov-mark

@makegov-mark makegov-mark Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

What

Turns the SDK's conformance check into a contract-first system that works out of the box and catches drift in both directions. Companion to makegov/tango#2589 (which adds type metadata to the contract).

  • Vendored contract at contracts/filter_shape_contract.json (schema_version: 2, generated from tango staging). New scripts/refresh_contract.py re-vendors from the ../tango sibling checkout or the GitHub API via gh.
  • Checker upgrades (scripts/check_filter_shape_conformance.py):
    • Staleness (new direction): a filter-like SDK argument the API no longer accepts is an error — it silently no-ops for users. Previously only the API→SDK direction was checked.
    • Types: each argument's annotation is validated against the contract's per-filter type metadata (date/choice/orderingstr, booleanbool, numberint/float/str; string is the API's catch-all and doesn't constrain).
    • Known-gaps baseline (contracts/conformance_baseline.json): accepted missing params downgrade to warnings — tracked backlog instead of silent drift. Removing the entry belongs in the PR that adds the param.
    • --suggest prints ready-to-paste typed parameter scaffolds for missing filters.
    • ordering is treated as accepted when the resource has ordering_fields/ordering_aliases (DRF ordering backends don't appear in FilterSets).
  • CI (lint.yml): conformance now runs unconditionally against the vendored contract — no token, works on forks. This closes the previous silent skip when TANGO_API_REPO_ACCESS_TOKEN was absent; the token now only powers a best-effort staleness notice comparing the vendored contract to tango HEAD.
  • pr_review.py conformance step defaults to the vendored contract (override with TANGO_CONTRACT_MANIFEST); docs updated (DEVELOPERS.md, scripts/README.md).

Why

SDK conformance was one-directional, name-only, and CI silently skipped it without a private-repo token. With the contract vendored and type-aware, drift between the tango API and this SDK is a hard, reproducible CI failure — in both directions — and new-filter work starts from generated scaffolds instead of archaeology.

Proof of value: the first run found 7 real coverage gaps the old setup missed (now baselined): key on contracts/IDVs/OTAs/OTIDVs, cage on entities, id on forecasts, opportunity_id on opportunities. Follow-up PR will close them via --suggest.

Testing

  • uv run python scripts/check_filter_shape_conformance.py → 0 errors, 11 warnings (7 baselined gaps + 4 unimplemented resources: events, news, offices, protests).
  • ruff format / ruff check / strict mypy tango/ clean; 332 unit tests pass. (tests/production/test_production_smoke.py::test_search_filters fails pre-existing on a clean tree — live-API smoke, unrelated.)
  • tests/test_shapes.py includes a one-line canonical ruff format fix picked up along the way.

Risks / notes

  • The staleness CI notice will warn until makegov/tango#2589 merges (the vendored contract is schema v2; tango HEAD's checked-in contract is still v1). Warning-only by design.
  • The vendored contract pins what this SDK conforms to; refresh cadence is manual for now (scripts/refresh_contract.py). A scheduled refresh PR job is a possible follow-up.

🤖 Generated with Claude Code

…line checks

- vendor canonical contract at contracts/filter_shape_contract.json
  (schema_version 2, generated from tango staging; refresh with new
  scripts/refresh_contract.py — sibling checkout or gh fallback)
- checker: staleness direction (SDK arg the API dropped = error),
  type conformance vs contract type metadata, known-gaps baseline
  (contracts/conformance_baseline.json), --suggest scaffold mode
- CI: conformance runs unconditionally against the vendored contract
  (closes the silent skip when TANGO_API_REPO_ACCESS_TOKEN is absent);
  token now only powers a staleness notice vs tango HEAD
- baseline 7 real coverage gaps found on first run: key (contracts/
  idvs/otas/otidvs), cage (entities), id (forecasts), opportunity_id
  (opportunities)
- docs: DEVELOPERS.md + scripts/README.md conformance sections;
  pr_review.py defaults to the vendored contract

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vdavez vdavez force-pushed the feat/contract-first-conformance branch from 3184966 to c5b8f44 Compare June 10, 2026 18:58
@vdavez vdavez merged commit dc9a2d3 into main Jun 10, 2026
12 checks passed
@vdavez vdavez deleted the feat/contract-first-conformance branch June 10, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant