diff --git a/CHANGELOG.md b/CHANGELOG.md index 0b89eb1..a7903cf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,50 @@ All notable changes to `views-frames` are documented here. The format is based o [Keep a Changelog](https://keepachangelog.com/), and this project adheres to [Semantic Versioning](https://semver.org/) as governed in `GOVERNANCE.md`. +## [1.8.1] — 2026-07-02 + +**Falsification-audit hardening (four-axis audit 2026-07-02; register C-67/C-68/C-69/C-70).** +Bug fixes with an identical contract — the code now honors what the docs already promised. +No public-surface change; `CONFORMANCE_FLOOR` stays `1.0.0`. + +### Fixed +- **`reconcile_proportional` conserves exactly for any nonzero draw sum (C-68, the audit's one + hard finding).** The torch-port's `+ 1e-8` denominator epsilon — a float32 no-op for draw sums + ≳ 0.1 but a **silent** deflator for tiny nonzero sums (a draw sum of 1e-8 reconciled a country + total of 100 to 50, with no error signal, violating the Reconcile.md §3 sum-to-country + guarantee) — is replaced by an explicit all-zero-draw guard: exact division for any nonzero + sum; all-zero draws stay zero exactly as before. **Bit-identical on all realistic data** + (torch-oracle parity unchanged and green). +- **Negative country totals now fail loud (C-68/F8):** `reconcile_proportional` raises + `ValueError` instead of silently clamping the output to zero (sum 0 ≠ the requested total). +- **The published conformance suites refuse to run under `python -O` (C-67):** all three + (`views_frames.conformance`, summarize, reconcile) now guard their entry points with + `_require_assertions()` — under optimized bytecode (which strips the suites' `assert` + statements) they raise `RuntimeError` instead of silently reporting green. +- **Empty-index `searchsorted` returns all `-1` (C-69)** — the documented not-found value — + instead of crashing with an obscure `IndexError` (the `np.clip(pos, 0, -1)` corner). + +### Notes +- Regression pins for all four fixes: `tests/test_falsification_safety_audit_2026_07.py`. +- `Reconcile.md` §6 documents the two reconcile behaviors; `proportional.py`'s module docstring + records the deliberate (bit-parity-preserving) deviation from the torch original. +- Register: C-67/C-68/C-69 registered-and-resolved; **C-70** (the audit's docs/tests polish + bundle) opened and **cleared in the same release** (below). + +### Tests +- **C-70 test adds** (#195): the share-**proportionality law** (the method's defining + forecast-proportion property, previously pinned only by the frozen-oracle fixtures); an mmap + **read-only pin** (`writeable is False`, in-place write raises); the reconcile + missing-`(time, priogrid_gid)`-mapping-entry raise. + +### Documentation +- **C-70 docs refresh** (#196): `CLAUDE.md` rewritten for the released three-package reality; + README banner → v1.8.0 + chronicle; ADR-013 as-built amendment (`feature_names`); CIC accuracy + fixes (§5 artifact names → `values.npy`, PredictionFrame §6 real dtype behavior, index §6 + NaN-via-dtype, Reconcile §10 pinning files); CICs/ADRs README framing refreshed; and a + **recurrence guard** — `validate_docs.sh` now checks the README banner's MAJOR.MINOR against + `pyproject.toml`. + ## [1.8.0] — 2026-06-28 **Native point-country broadcast in `views_frames_reconcile` (ADR-023 amendment, #143 / Epic #142), @@ -43,7 +87,7 @@ additive — the frozen leaf and summarize public surface are unchanged, and the was the last non-trivial surface without a CIC (ADR-006); **register C-64 resolved**. - **ADR-025 — value-buffer immutability is by convention; only the index is enforced** (Epic #179). Corrects the "immutable value objects" contract (the three frame CICs §9/§3 + README design - principle 2) to match the code: the index (`time`/`unit`) is `setflags(write=False)`-enforced; the + principle 3) to match the code: the index (`time`/`unit`) is `setflags(write=False)`-enforced; the value buffer is immutable *by convention* (left writeable to preserve zero-copy / `mmap` — mutating `.values` in place is unsupported). The `setflags`-enforce on `.values` would be a MAJOR ("tightening an invariant" on a frozen-surface member, GOVERNANCE/ADR-018), so it is recorded as a diff --git a/CLAUDE.md b/CLAUDE.md index dc796a0..4cb0293 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,34 +1,54 @@ # views-frames The VIEWS platform's **data-contract layer**: small, stable, abstract, immutable -array+identifier value objects (`FeatureFrame`, `PredictionFrame`, `TargetFrame`, -and anticipated siblings) at the **root of the platform dependency DAG**. numpy -only; depends on nothing internal; every other repo depends *toward* it. +array+identifier value objects (`PredictionFrame`, `TargetFrame`, `FeatureFrame`) +at the **root of the platform dependency DAG**, plus two sibling operation packages +in the same wheel. numpy only; depends on nothing internal; every other repo +depends *toward* it. -> **Status:** **implemented — v0.1.0** (Epic 2). `src/views_frames/` realises the -> contract (index, frames, io, conformance suite). Consumer adoption (re-export -> shims, pandas migration) is Epic 3. +> **Status:** **released — v1.8.x on PyPI**, public API **frozen since v1.0.0** +> (ADR-018; everything after is additive, `CONFORMANCE_FLOOR` stays `1.0.0`). +> Consumers install `views-frames` and validate against the published conformance +> suite (`views_frames.conformance`, ADR-016). See `CHANGELOG.md` for the release +> history and `README.md` §status for the version chronicle. ## Architecture -**Two packages** under `src/` (datafactory multi-package pattern), strict one-way -dependency `views_frames_summarize → views_frames`, enforced by -`tests/test_import_enforcement.py`: +**Three packages** under `src/` (one wheel), strict one-way dependencies +`views_frames_summarize → views_frames` and `views_frames_reconcile → views_frames` +(siblings never import each other), enforced by `tests/test_import_enforcement.py`: -**`src/views_frames/`** — the pure data contract (numpy-only; depends on nothing): +**`src/views_frames/`** — the pure data contract (numpy-only; depends on nothing; frozen): -- `index.py` — `SpatioTemporalIndex` (`{time, unit, level}` + same-level numpy alignment). +- `index.py` — `SpatioTemporalIndex` (`{time, unit, level}`; same-level numpy alignment + + consumer-injected cross-level remap; identifier arrays write-protected). - `spatial_level.py` — `SpatialLevel` (cm/pgm identifier vocabulary; labels only). -- `protocols.py` — `Frame` / `SpatioTemporalIndexed` / `Sampled` (`sample_count`/`is_sample` only) / `Persistable`. +- `protocols.py` — `Frame` / `SpatioTemporalIndexed` / `Sampled` (`sample_count`/`is_sample` + only) / `Persistable` (four small segregated protocols). +- `metadata.py` — `FrameMetadata` (typed, frozen, generic-only provenance; ADR-020). - `_validation.py` — shared construction-time invariants. -- `feature_frame.py`, `prediction_frame.py`, `target_frame.py` — sibling frames (no shared base; ADR-011 Option C). -- `io/` — `npz` (native) + `arrow` (flat-columnar). **The only place `pyarrow` may be imported.** +- `feature_frame.py`, `prediction_frame.py`, `target_frame.py` — sibling frames + `(N,F,S)` / `(N,S)` / `(N,1)` (no shared base; ADR-011 Option C). +- `io/` — `npz` (native, mmap-capable) + `arrow` (flat-columnar parquet codec, + module-level by decision D-11). **The only place `pyarrow` may be imported.** +- `conformance/` — the published suite consumers run in *their* CI + (`assert_frame_contract`, `assert_frame_envelope`, alignment laws; `CONFORMANCE_FLOOR`). **`src/views_frames_summarize/`** — sample-axis posterior summarization *over* frames -(ADR-017; numpy-only; depends on `views_frames`). Point estimates -(`collapse(frame, reducer)`, `map_estimate`) return a `(N,…,1)` frame; intervals -(`hdi`, `quantiles`) return arrays aligned to the frame's index. **Never** owns IO, -domain data, scoring, or reconciliation. +(ADR-017; numpy-only; depends on `views_frames`). Point estimates (`collapse`, +`map_estimate`, `tower_point`) return `(N,…,1)` frames; intervals/arrays (`hdi`, +`quantiles`, `hdi_tower`, `exceedance`, `expected_shortfall`, `bimodality`) return +index-aligned arrays; `aggregate_distributions[_arrays]` sums sample distributions +across levels (joint sampling). Fail-loud config for the tower family (ADR-019). +**Never** owns IO, domain data, scoring, or reconciliation. + +**`src/views_frames_reconcile/`** — top-down proportional reconciliation of pgm grid +forecasts to cm country totals (ADR-023; numpy-only; depends on `views_frames`). +`ReconciliationModule(map_keys, map_vals)` with the geography **injected** (never +fetched); `reconcile` / `reconcile_result` (the latter reports the +`point-broadcast`/`aligned-draws` **mode** — returned, never stamped on the leaf +header, D-12). The per-draw method is a documented approximation; the principled +joint upgrade is designed and deferred (ADR-024, register C-62). ## Tooling (uv + hatchling) @@ -36,32 +56,46 @@ Always invoke via `uv run`: ```bash uv sync # install deps + the package (editable) -uv run pytest # tests (incl. import-enforcement + falsification stubs) +uv run pytest # tests (incl. import-enforcement + falsification suites) uv run ruff check . # lint uv run ruff format . # format -uv run mypy src/ # type check (strict) +uv run mypy src/ # type check (strict; also check with --python 3.11 before pushing) uv build # build wheel + sdist ``` +CI additionally gates 100% line+branch coverage +(`uv run pytest --cov --cov-fail-under=100`) and a numpy-floor job. + ## Design principles (the hard constraints) 1. **numpy only in the core.** Never import `pandas`, `polars`, `geopandas`, - `wandb`, `viewser`, `torch`, or any `views_*` package. `pyarrow` is allowed + `wandb`, `viewser`, `torch`, or any foreign `views_*` package. `pyarrow` is allowed *only* under `io/`. Enforced by `tests/test_import_enforcement.py` (ADR-002). 2. **Immutable value objects.** Operations return new frames; structural ops share - the buffer (zero-copy); only reductions allocate (ADR-013, register C-07). -3. **Fail loud at construction.** Invariants raise `ValueError`/`TypeError`; the - guarantee is *structural*, not temporal (`time` is opaque; register C-11). + the buffer (zero-copy); only reductions allocate (register C-07). Enforced for the + *index*; **by convention** for the value buffer (writeable on purpose to preserve + zero-copy/mmap — mutating `.values` in place is unsupported; ADR-025, C-66 rider). +3. **Fail loud.** Invariants raise `ValueError`/`TypeError` at construction and at + every validation guard; the guarantee is *structural*, not temporal (`time` is + opaque; register C-11). 4. **No shared frame base.** Frames are separate siblings (ADR-011); cm/pgm is a `SpatialLevel` *value*, never a class axis. -5. **No domain data.** Cross-level cm↔pgm alignment is a protocol with a - consumer-injected mapping; the leaf never embeds/fetches it (ADR-014). -6. **One concept per file**; explicit `__init__.py` re-exports (no `import *`). +5. **No domain data.** Cross-level cm↔pgm alignment takes a consumer-injected + mapping; the leaf and siblings never embed/fetch it (ADR-014/ADR-023). +6. **One concept per file** (test-enforced: ≤1 public class per module); explicit + `__init__.py` re-exports (no `import *`). +7. **Frozen surface + WET before DRY.** The v1 public surface only grows additively + (removal/tightening = MAJOR + cross-repo coordinated bump, GOVERNANCE.md); + deliberate duplication recorded in ADRs is a choice, not debt. ## Governance -Constitutional ADRs 000–010, project ADRs 011–016, CIC infrastructure, contributor -protocols, and standards live in `docs/`. The technical risk register is -maintained as an internal governance artifact. Run `bash docs/validate_docs.sh` to check -documentation consistency. Build *against* the README design bible — if code and -README disagree, reconcile before merging. +Constitutional ADRs 000–010, project ADRs 011–025, CICs for every non-trivial +surface (7 active incl. the package-level `Summarize.md` and `Reconcile.md`), +contributor protocols, and standards live in `docs/`. The technical risk register +(`reports/technical_risk_register.md`) is the curated concern/decision log. Run +`bash docs/validate_docs.sh` to check documentation consistency. Build *against* +the README design bible — if code and README disagree, reconcile before merging. +Releases: dev→main via **merge commit** (never squash — main carries squash release +commits dev lacks), then `gh release create vX.Y.Z` triggers the PyPI publish +(Trusted Publishing; see `docs/guides/publishing-to-pypi.md`). diff --git a/README.md b/README.md index 1a05f95..be9c706 100644 --- a/README.md +++ b/README.md @@ -4,14 +4,18 @@ > containers (`FeatureFrame`, `PredictionFrame`, and their anticipated siblings) > that every other repo depends on and that depends on nothing internal. > -> **Status:** **v1.7.0 — frozen API** (frozen since v1.0.0, ADR-018; the v1.1 surface is +> **Status:** **v1.8.0 — frozen API, published to PyPI** (frozen since v1.0.0, ADR-018; the +> v1.1 surface is > purely additive — the coherent posterior summary, ADR-019; v1.2.0 rebuilt the tower > `outside-in`, C-44; v1.3.0 makes the tower summary distribution-agnostic — no magnitude > zeroing by default, register C-45; v1.4.0 adds generic provenance to `FrameMetadata` > (`run_id`/`data_version`) and publishes the shared `assert_frame_envelope` checker, > ADR-020; v1.5.0 adds the threshold **exceedance** estimator `P(Y > c)`, ADR-021; v1.6.0 > adds the worst-case **expected_shortfall** estimator, ADR-022; v1.7.0 adds a third -> sibling package **`views_frames_reconcile`** — forecast reconciliation, ADR-023). This +> sibling package **`views_frames_reconcile`** — forecast reconciliation, ADR-023; v1.8.0 +> adds the native point-country broadcast + the self-describing reconciliation mode, +> the three showcase notebooks, and the frames-family hardening pass — Reconcile.md CIC, +> ADR-025 immutability-by-convention, the adversarial red-test batch). This > README is the design > bible; the contract it specifies is realised in `src/views_frames/` (index, frames, > io, conformance suite) plus the `src/views_frames_summarize/` sibling package diff --git a/docs/ADRs/013_metadata_and_identifier_model.md b/docs/ADRs/013_metadata_and_identifier_model.md index 38e6852..33051b5 100644 --- a/docs/ADRs/013_metadata_and_identifier_model.md +++ b/docs/ADRs/013_metadata_and_identifier_model.md @@ -81,6 +81,17 @@ the index aligns rows; the header describes the frame. ## Implementation Notes +> **As-built amendment (2026-07-02, four-axis audit / register C-70).** Two details landed +> differently from the notes below, both consistent with the decision's substance: +> (1) **`feature_names` is not a header field** — it is a `FeatureFrame` constructor +> argument/attribute (`feature_frame.py`), validated there against the feature axis +> (`len(feature_names) == values.shape[1]`) and serialized with the frame; `FrameMetadata` +> stays purely generic provenance (which ADR-020 later ratified as generic-*only*). +> (2) "Validated at construction" applies to the **frames and identifiers**, not the header +> fields themselves — `FrameMetadata` is a frozen dataclass of all-optional fields; +> `from_dict` tolerates unknown keys (forward-compatible). The typed-optional-extensible +> header and fixed `{time, unit}` identifiers — the decision's core — are implemented as decided. + - Model the header as a frozen dataclass (`FrameMetadata`) with all-optional fields + validation; `feature_names` lives here for `FeatureFrame`. - `_validation.py` enforces required identifiers `{time, unit}`; optional identifiers validated diff --git a/docs/ADRs/025_value_buffer_immutability_by_convention.md b/docs/ADRs/025_value_buffer_immutability_by_convention.md index a716072..df21bc9 100644 --- a/docs/ADRs/025_value_buffer_immutability_by_convention.md +++ b/docs/ADRs/025_value_buffer_immutability_by_convention.md @@ -50,7 +50,7 @@ index is write-protected.** new frame instead. This is the documented contract; it is **not** mechanically enforced. 2. The leaf deliberately leaves the value buffer **writeable** so that structural/metadata operations stay **zero-copy** and `mmap`-backed frames stay `mmap`-backed (design - principle 2 / C-07). The enforced invariant is on the **index** (`time`/`unit`), which is + principle 3 / C-07). The enforced invariant is on the **index** (`time`/`unit`), which is small, never shared mutably, and already `setflags(write=False)`. 3. **The `setflags`-enforce on `.values` is a deferred MAJOR-rider.** When a MAJOR bump happens for any reason, enforcement is added **for free** as part of that coordinated bump: @@ -58,7 +58,7 @@ index is write-protected.** test that `frame.values.flags.writeable is False`. It is **out of scope** to do this on its own (it does not justify a standalone MAJOR). -In scope: correcting the docs (the three frame CICs, README principle 2) to state the +In scope: correcting the docs (the three frame CICs, README principle 3) to state the by-convention reality. Out of scope: any code change; any change to `CONFORMANCE_FLOOR` (stays `1.0.0`); enforcing read-only `.values` now. diff --git a/docs/ADRs/README.md b/docs/ADRs/README.md index baf4184..c288b05 100644 --- a/docs/ADRs/README.md +++ b/docs/ADRs/README.md @@ -2,16 +2,17 @@ # ADR README and Governance Map — views-frames This repository uses Architecture Decision Records (ADRs) to govern structural, semantic, -and operational behavior. Because `views-frames` is currently a **design bible** (a -README, consumer-review findings, design critiques, and falsification stubs — no `src/` yet), these -ADRs describe the **intended** architecture; when the leaf is stood up, code must conform -to them (and to the README, which is itself authoritative). +and operational behavior. The architecture these ADRs describe is **implemented and +released** (three packages under `src/`, public API frozen since v1.0.0); code must +conform to them (and to the README design bible, which is itself authoritative — if code +and README disagree, reconcile before merging). ADRs are divided into: 1. **Constitutional ADRs (000–009)** — foundational architectural rules. 2. **Governance ADRs (010)** — the technical risk register. -3. **Project-Specific ADRs (011–016)** — the ratified contract decisions. +3. **Project-Specific ADRs (011–025)** — the ratified contract, estimator, sibling-package, + and freeze/immutability decisions. --- @@ -50,10 +51,13 @@ Together, these define the invariant layer of the system. --- -## Project-Specific ADRs (011–016) — the ratified contract decisions +## Project-Specific ADRs (011–025) — the ratified project decisions -These ratify the six resolved decisions from the design bible (README §13a). All **Accepted** -2026-06-21; each cites the risk-register IDs it resolves. +**011–016** ratify the six founding contract decisions from the design bible (README §13a, +all Accepted 2026-06-21); **017–025** are the post-v1 decisions — sibling packages, the API +freeze, estimators, contract homes, and the immutability convention (Accepted 2026-06-22 +through 2026-07; ADR-024 is a design-direction ADR with implementation deferred). Each cites +the risk-register IDs it resolves. - **ADR-011** — [Twin-unification model (Option C)](011_twin_unification_option_c.md). Share only `SpatioTemporalIndex` + `_validation` + protocols + `io/`; relocate the frames as separate sibling classes; reject the `_BaseFrame` god-class (A); defer the composed header (B). Resolves D-03; relates to C-16, C-03. - **ADR-012** — [Sample axis convention](012_sample_axis_convention.md). Always an explicit trailing axis (`S ≥ 1`); `is_sample = S > 1`; `collapse` reduces the trailing axis; one shape contract. Relates to C-02, C-16, C-17. diff --git a/docs/CICs/FeatureFrame.md b/docs/CICs/FeatureFrame.md index ce1cdbe..9da00f9 100644 --- a/docs/CICs/FeatureFrame.md +++ b/docs/CICs/FeatureFrame.md @@ -3,7 +3,7 @@ **Status:** Active **Owner:** VIEWS platform maintainers -**Last reviewed:** 2026-06-24 +**Last reviewed:** 2026-07-02 **Related ADRs:** ADR-001, ADR-008, ADR-011, ADR-012, ADR-013 > Implemented in v0.1.0 (`src/views_frames/feature_frame.py`). This contract governs @@ -59,7 +59,7 @@ Violations raise at construction (ADR-008). ## 5. Outputs and Side Effects -- New frames from operations; `save` writes `y_features.npy` + `identifiers.npz` +- New frames from operations; `save` writes `values.npy` + `identifiers.npz` + `feature_names`/`metadata` (via the frame-state round-trip contract). --- diff --git a/docs/CICs/PredictionFrame.md b/docs/CICs/PredictionFrame.md index 1c8cedd..e623213 100644 --- a/docs/CICs/PredictionFrame.md +++ b/docs/CICs/PredictionFrame.md @@ -3,7 +3,7 @@ **Status:** Active **Owner:** VIEWS platform maintainers -**Last reviewed:** 2026-06-24 +**Last reviewed:** 2026-07-02 **Related ADRs:** ADR-001, ADR-008, ADR-011, ADR-012, ADR-013, ADR-017 > Implemented in v0.1.0 (`src/views_frames/prediction_frame.py`), relocated from @@ -61,16 +61,19 @@ Violations raise at construction (ADR-008) — never log-and-continue. ## 5. Outputs and Side Effects -- New frames from operations; `save` writes `y_pred.npy` + `identifiers.npz` +- New frames from operations; `save` writes `values.npy` + `identifiers.npz` (+ header). No other side effects. --- ## 6. Failure Modes and Loudness -- Raises `TypeError` on non-`float32`/object-dtype values; `ValueError` on shape, +- Raises `ValueError` on object-dtype values (list-in-cell is banned), on shape, length, or completeness violations, and on `reindex(other)` when `other` is not a - subset of this frame's index. The structural guarantee is **not temporal** (register C-11). + subset of this frame's index. Other numeric dtypes (e.g. float64) are **coerced** + to float32 by copy at construction — accepted, not rejected (the no-copy fast path + is float32-only, register C-07). The structural guarantee is **not temporal** + (register C-11). --- diff --git a/docs/CICs/README.md b/docs/CICs/README.md index 30ff9c4..995a687 100644 --- a/docs/CICs/README.md +++ b/docs/CICs/README.md @@ -49,17 +49,18 @@ Contracts must be clear enough that: --- -## Status: infrastructure only (no contracts yet) +## Status: fully contracted -`views-frames` currently contains no code — it is a design bible (see `README.md`). This -directory holds the template (`cic_template.md`) and this README. **Contracts are authored -as each class is implemented**, when the twins are relocated and the leaf is stood up. +Every non-trivial surface across the three shipped packages (`views_frames`, +`views_frames_summarize`, `views_frames_reconcile`) is governed by an active CIC below. +New contracts are authored **with** the class/package that introduces them (the +`Reconcile.md` gap was the last, closed 2026-06-28, register C-64). --- ## Active Contracts -These CICs govern the classes implemented in v0.1.0 (`src/views_frames/`). +These CICs govern the shipped surface (`src/`, three packages, frozen since v1.0.0). - `SpatioTemporalIndex.md` — the genuinely-reused alignment primitive; same-level logic owned, cross-level mapping injected (ADR-014). diff --git a/docs/CICs/Reconcile.md b/docs/CICs/Reconcile.md index 5341e44..8565a93 100644 --- a/docs/CICs/Reconcile.md +++ b/docs/CICs/Reconcile.md @@ -149,10 +149,16 @@ All input inconsistencies **raise `ValueError`** before any scaling work → raises (`"map_keys must be"` / `"map_vals must be"`). - A grid row whose `(time, priogrid_gid)` is absent from the injected mapping → raises (via `cross_level_align`). +- A **negative country total** → raises (`"country totals must be non-negative"`) — it cannot + be conserved under the non-negativity clamp, so silently clamping to zero would break the + sum-to-country guarantee (falsify audit 2026-07, F8). **Never silent:** a level/coverage/shape inconsistency must surface as a `ValueError`, not a mid-compute crash or a wrong-but-plausible frame. **The all-zero-country draw is an edge, not a failure:** it stays `0` (no mass to distribute), not `NaN` and not the country total. +**Sum-to-country holds exactly for any nonzero draw sum** — the port's `+ 1e-8` denominator +epsilon, which silently deflated the scale factor for tiny nonzero sums, was replaced by an +explicit all-zero guard (falsify audit 2026-07, F1; bit-identical on all realistic data). **The per-draw approximation is loud-by-documentation, not by exception (C-62):** an `aligned-draws` result of independently-trained models is a documented approximation @@ -249,14 +255,20 @@ gate); these are the contract-bearing suites: both-points, pre-tiled cm) — `TestReconciliationResult`. - **Red (it fails loud):** `tests/test_reconciliation_validation.py` — `validate_reconciliation_inputs` raises on wrong cm level, wrong grid level, sample-count - mismatch, time mismatch, and a missing country forecast; the `(M, 2)` map-keys guard - (`test_bad_mapping_shape_raises`). The conformance-suite **negative** (a deliberately - non-conforming impl/input makes `assert_reconcile_contract` raise) and the - `ReconciliationResult` frozen-ness assertion are the red gaps being closed alongside this - contract (register C-65 batch, epic #179 / S3). - -Each §3 guarantee maps to a green/beige test; each §6 failure mode maps to a red test in -`test_reconciliation_validation.py`. + mismatch, time mismatch, a missing country forecast, and a missing + `(time, priogrid_gid)` mapping entry (`test_missing_mapping_entry_raises`); the `(M, 2)` + map-keys guard (`test_bad_mapping_shape_raises`). The conformance-suite **negative** (a + deliberately non-conforming impl makes `assert_reconcile_contract` raise) and the + `ReconciliationResult` frozen-ness assertion live in `test_reconcile_conformance.py` / + `test_reconciliation_e2e_parity.py` (the C-65 batch, epic #179 / S3). The + share-proportionality **law** (`test_shares_are_preserved_within_each_draw`) and the + 2026-07 audit regressions — exact conservation for tiny nonzero sums, the negative-total + raise — are pinned in `test_reconciliation_parity.py` and + `test_falsification_safety_audit_2026_07.py` (register C-68/C-70). + +Each §3 guarantee maps to a green/beige test; each §6 failure mode maps to a red test +(the validation guards in `test_reconciliation_validation.py`; the negative-total and +tiny-sum modes in the audit-regression file above). --- diff --git a/docs/CICs/SpatioTemporalIndex.md b/docs/CICs/SpatioTemporalIndex.md index 28387e0..5ddfd60 100644 --- a/docs/CICs/SpatioTemporalIndex.md +++ b/docs/CICs/SpatioTemporalIndex.md @@ -77,8 +77,11 @@ Assumptions that do not hold **must raise** at construction (ADR-009), never fal ## 6. Failure Modes and Loudness -- Raises `TypeError` if any identifier array is not integer dtype. -- Raises `ValueError` if arrays differ in length, contain NaN, or `level` is invalid. +- Raises `TypeError` if any identifier array is not integer dtype — which is also what + a NaN-carrying array hits (NaN forces a float dtype; integers cannot be NaN, so the + "no NaN" guarantee is enforced *via* the dtype check, not a separate `ValueError`). +- Raises `ValueError` if arrays differ in length; `TypeError` if `level` is not a + `SpatialLevel`. - `cross_level_align` raises if called without an injected mapping — it must **never** silently fetch or assume one (the defining boundary of ADR-014). - Nothing fails silently. diff --git a/docs/CICs/Summarize.md b/docs/CICs/Summarize.md index adf692f..958597a 100644 --- a/docs/CICs/Summarize.md +++ b/docs/CICs/Summarize.md @@ -3,7 +3,7 @@ **Status:** Active **Owner:** VIEWS platform maintainers -**Last reviewed:** 2026-06-24 +**Last reviewed:** 2026-07-02 **Related ADRs:** ADR-002, ADR-008, ADR-009, ADR-011, ADR-012, ADR-014, ADR-017, ADR-019, ADR-021 > The `views_frames_summarize` package (functions, not a class). Implemented in diff --git a/docs/CICs/TargetFrame.md b/docs/CICs/TargetFrame.md index 6764704..00956eb 100644 --- a/docs/CICs/TargetFrame.md +++ b/docs/CICs/TargetFrame.md @@ -3,7 +3,7 @@ **Status:** Active **Owner:** VIEWS platform maintainers -**Last reviewed:** 2026-06-24 +**Last reviewed:** 2026-07-02 **Related ADRs:** ADR-001, ADR-008, ADR-011, ADR-012, ADR-013 > Implemented in v0.1.0 (`src/views_frames/target_frame.py`). This contract governs @@ -56,7 +56,7 @@ Violations raise at construction (ADR-008). ## 5. Outputs and Side Effects -- New frames from operations; `save` writes `y_true.npy` + `identifiers.npz`. No other +- New frames from operations; `save` writes `values.npy` + `identifiers.npz`. No other side effects. --- diff --git a/docs/validate_docs.sh b/docs/validate_docs.sh index 8bd3b9b..6f24d86 100755 --- a/docs/validate_docs.sh +++ b/docs/validate_docs.sh @@ -84,6 +84,26 @@ echo "--- Checking template status markers ---" template_count=$(grep -rl '\-\-template\-\-' --include='*.md' . 2>/dev/null | wc -l) echo " INFO: $template_count files still have --template-- status (expected in template repo)" +# 6. README status banner tracks the released MAJOR.MINOR (guards the narrative +# epoch-lag found by the 2026-07 audit, register C-70: the banner sat at v1.7.0 +# while 1.8.0 was on PyPI). Patch releases are exempt — the banner tracks the +# surface, which only MINORs change. +echo "--- Checking README status banner vs pyproject version ---" +if [ -f "../pyproject.toml" ] && [ -f "../README.md" ]; then + pyver=$(grep -m1 '^version = ' ../pyproject.toml | sed 's/version = "\(.*\)"/\1/') + pymm=$(echo "$pyver" | cut -d. -f1,2) + banner=$(grep -m1 -oE '\*\*Status:\*\* \*\*v[0-9]+\.[0-9]+' ../README.md | grep -oE '[0-9]+\.[0-9]+') + if [ -z "$banner" ]; then + echo " ERROR: could not find a '**Status:** **vX.Y' banner in README.md" + errors=$((errors + 1)) + elif [ "$banner" != "$pymm" ]; then + echo " ERROR: README banner says v$banner but pyproject version is $pyver (MAJOR.MINOR $pymm)" + errors=$((errors + 1)) + else + echo " OK (banner v$banner ~ pyproject $pyver)" + fi +fi + echo "" if [ "$errors" -gt 0 ]; then echo "=== FAILED: $errors issue(s) found ===" diff --git a/pyproject.toml b/pyproject.toml index dc5a04f..faeb7b2 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "views-frames" -version = "1.8.0" +version = "1.8.1" description = "The VIEWS platform data-contract layer: immutable array+identifier frames (numpy only, root of the dependency DAG)." authors = [ { name = "Simon Polichinel von der Maase", email = "simmaa@prio.org" }, diff --git a/reports/technical_risk_register.md b/reports/technical_risk_register.md index 9cbb600..22a8c81 100644 --- a/reports/technical_risk_register.md +++ b/reports/technical_risk_register.md @@ -4,10 +4,10 @@ |-------------------|--------------------------------------| | Project | views-frames | | Owner | VIEWS platform maintainers | -| Last Updated | 2026-06-27 | -| Total Concerns | 62 | -| Open Concerns | 11 | -| Resolved Concerns | 51 | +| Last Updated | 2026-07-02 | +| Total Concerns | 67 | +| Open Concerns | 12 | +| Resolved Concerns | 55 | | Disagreements | 12 | --- @@ -35,17 +35,17 @@ > and formalised by ADRs 011–016, all of which merged and shipped/froze in **v1.0.0** > (ADR-018) — they are now in **Resolved Concerns**. C-01/C-08/C-12 are resolved-by-decision > and persist only as **frozen-invariant guards** (their triggers protect the frozen scope). -> The **12 currently open** concerns fall into four clusters plus a cross-cutting theme +> The **12 currently open** concerns fall into five clusters plus a cross-cutting theme > (detailed under *Causal clusters* in Register Conventions): **(1) summarize-estimator > coherence (#89)** — C-32, C-34, C-43, C-57 (the under-determined frozen MAP/bimodality > estimators); **(2) reconcile method + governance** — C-58, C-62 (+ D-12): a pragmatic > per-draw port with a deferred principled upgrade (ADR-024); the package's missing CIC (C-64) > was closed by `Reconcile.md` (epic #179 / S1); **(3) > construction-convenience accretion (#113)** — C-52, C-53, C-54 (+ D-09), the `from_arrays` -> "camel's nose"; **(4) cross-repo coordination** — C-13, C-46 (+ D-04/D-05/D-06). The -> former **immutability-enforcement** cluster (C-63) is resolved: the contract was corrected -> to the by-convention reality and the `setflags`-enforce deferred to a future MAJOR (ADR-025, -> epic #179 / S2). The cross-cutting +> "camel's nose"; **(4) cross-repo coordination** — C-13, C-46 (+ D-04/D-05/D-06); **(5) +> immutability enforcement** — C-66 (+ resolved C-63): the C-63 contract-correction is done +> (ADR-025, epic #179 / S2), but the actual `setflags`-enforce on `.values` is a MAJOR and is +> deferred — **C-66** tracks that enforce-rider so it stays visible in Open. The cross-cutting > **verification-completeness** theme now narrows to C-58 (the reconciler's production-slice > check); its sibling C-65 (non-finite fail-loud on the blocked path) was pinned with a red > test (epic #179 / S3). Earlier clusters are closed: @@ -222,6 +222,21 @@ The Epic 11 cutover was validated by a sound transitive chain — `new == old vp --- +### C-66: value-buffer write-protection is deferred to the next MAJOR (the C-63 enforce-rider) + +| Field | Value | +|-------|-------| +| ID | C-66 | +| Tier | 3 | +| Source | review-diff + register-risk (2026-06-28, epic #179 / S2) — the residual of the C-63 resolution-by-decision (ADR-025). | +| Trigger | When a MAJOR bump is opened for **any** reason — add `self._values.setflags(write=False)` after the `self._values = ...` assignment in the three frame constructors (`prediction_frame.py:44`, `target_frame.py:43`, `feature_frame.py:52`) **and** a red test (`frame.values.flags.writeable is False`; mirror `tests/test_properties.py:38`), riding that MAJOR for free. **Or** sooner, if a consumer is found applying an in-place `.values` mutation (`frame.values[mask] = 0`, `*=`, a clamp) on a `with_metadata`/`select` buffer-sharing frame — promote/expedite the enforce then. | +| Location | `src/views_frames/{prediction_frame.py:44, target_frame.py:43, feature_frame.py:52}` (bare `self._values = values`, no `setflags`); `src/views_frames/_validation.py:64` (`coerce_values` returns float32 without copy); contrast `src/views_frames/index.py:55-56` (the index **is** write-protected). Decision in `docs/ADRs/025_value_buffer_immutability_by_convention.md`. | +| Cross-refs | **C-63** (RESOLVED by contract correction — this entry tracks the *deferred enforce* it left open), **ADR-025** (the decision + the exact one-line-per-constructor change), ADR-018 (`values` is frozen-surface, so the enforce is a MAJOR), GOVERNANCE.md (SemVer: "tightening an invariant" = MAJOR), C-07 (the zero-copy reason the buffer is left writeable). | + +C-63 was resolved by **correcting the contract** (ADR-025): the value buffer is documented as immutable *by convention* and the docs no longer claim an unenforced guarantee. But the **code** is unchanged — `frame.values.flags.writeable` is still `True`, and `with_metadata` shares the buffer — so the underlying mechanism (an in-place `.values` mutation **silently corrupts every frame sharing the buffer**, the Tier-2 basis of C-63) is **mitigated, not removed**. The mitigation is documentation (three frame CICs §9 + README design principle 3 say it is unsupported) + the empirical fact that **nothing in `src/` or `tests/` mutates `.values`**. The actual write-protection (`setflags(write=False)`) is deliberately deferred because, on the frozen-surface `values`, it is a **MAJOR** (GOVERNANCE/ADR-018) and does not justify a standalone cross-repo coordinated bump. **Tier 3** — this entry tracks the *accepted deferral* of a documented-and-unexercised exposure (the acute silent-corruption path requires a consumer to ignore the published contract); it is a governance/safety-tracking item, not a current defect, and exists so the deferred enforce stays visible in the **Open** section rather than buried in a resolved entry. **Open** — until the enforce rides the next MAJOR. + +--- + ## Disagreements ### D-01: `SpatioTemporalIndex` domain-purity fork (where does cross-level alignment live?) @@ -362,6 +377,66 @@ Cross-refs: C-47 (eval provenance kept out of the generic header — the precede ## Resolved Concerns +### C-70: audit polish bundle — docs narrative epoch-lag + three small test adds (four-axis audit 2026-07) — RESOLVED + +| Field | Value | +|-------|-------| +| ID | C-70 | +| Tier | 3 | +| Source | four-axis audit 2026-07-02 (review-base-docs Phase 2 + test-review Phase 3) | +| Trigger | The next docs or tests PR — fold this bundle in rather than letting the narrative lag compound (a new contributor/agent onboarding from `CLAUDE.md` today is told a two-package v0.1.0 architecture). | +| Location | **Docs:** `CLAUDE.md` (epoch-stale: "Two packages", "Status: v0.1.0", no `views_frames_reconcile`); `README.md:7` (banner says v1.7.0; chronicle ends at 1.7.0); `docs/ADRs/013_*.md` (claims `feature_names` lives in `FrameMetadata` — it is a `FeatureFrame` constructor arg, `feature_frame.py:32`); `docs/CICs/{PredictionFrame,TargetFrame,FeatureFrame}.md` §5 (say `y_pred/y_true/y_features.npy` — actual artifact is `values.npy`, `io/npz.py:34`); `PredictionFrame.md` §6 ("TypeError on non-float32" — float64 is *coerced*, object dtype raises *ValueError*); `docs/CICs/README.md:52-56` ("no contracts yet" contradicting its own Active list); `docs/ADRs/README.md:6-55` (design-bible framing, "011–016", "six decisions"); `Reconcile.md` §10 (blanket "each §6 mode maps to a red test" — the missing-map-entry mode is pinned only at the leaf); stale `Last reviewed` dates on 3 frame CICs. **Tests:** a share-proportionality *law* test (the method's essence is otherwise pinned only by the frozen oracle; the falsify F5 probe proved the law holds — commit it); an mmap read-only pin (`frame.values.flags.writeable is False` after `load(mmap=True)` — F2 proved it); a reconcile-suite test for the missing-`(time,gid)`-mapping-entry raise. | +| Cross-refs | The systemic pattern: all doc drift is in narrative text `validate_docs.sh` does not check (banners, framing, filenames), while every mechanically-validated element is current. C-46 (the "verification surface depends on usage" family), C-58 (test-realism, registered), resolved C-67/C-68/C-69 (the fixed half of the same audit). | + +The 2026-07 four-axis audit found the code↔contract agreement strong (30/30 public symbols CIC-covered, 15/16 project ADRs accurate, ~57/60 CIC guarantee items pinned) but the **narrative documentation an epoch behind** and three cheap test additions open. None affects correctness; the CLAUDE.md/ADR-013 items are the material ones (they misinform onboarding). **Tier 3** — maintainability/onboarding accuracy, multiple contributors affected via CLAUDE.md. **Resolved** (2026-07-02, the option-3 cleanup): the tests half by PR #195 (share-proportionality law, mmap read-only pin, missing-map-entry raise); the docs half in the follow-up docs PR (CLAUDE.md rewritten for the three-package released reality; README banner → v1.8.0 + chronicle; ADR-013 as-built amendment; the three CIC §5 filenames → `values.npy`; PredictionFrame §6 coercion wording; SpatioTemporalIndex §6 NaN-via-dtype wording; CICs/ADRs README framing refreshed; `Last reviewed` dates bumped; Reconcile.md §10 updated to name the actual pinning files). Plus a **recurrence guard**: `validate_docs.sh` now checks the README banner's MAJOR.MINOR against `pyproject.toml`, so the narrative epoch-lag pattern fails validation instead of accumulating. + +--- + +### C-68: `reconcile_proportional` silently violated sum-to-country for tiny nonzero draw sums (the `+ 1e-8` epsilon), and silently clamped negative totals — RESOLVED + +| Field | Value | +|-------|-------| +| ID | C-68 | +| Tier | 2 | +| Source | test-review (2026-07-02, the untested-region flag) + falsify audit 2026-07-02 (F1 hard, F8 soft — empirically confirmed, independently reproduced) | +| Trigger | (historical) A country-month whose nonzero grid draws summed ≲1e-4 against a material cm total — the reconciled output silently under-conserved (50% shortfall at draw-sum 1e-8, 91% at 1e-9; still violating the §3 rtol at 1e-5), and `assert_reconcile_contract` rejected the package's own output. A negative country total silently clamped to an all-zero output (sum 0 ≠ the total). | +| Location | `src/views_frames_reconcile/proportional.py` (formerly `_EPS = 1e-8` at :30, `sum_nonzero + _EPS` at :74). | +| Cross-refs | Reconcile.md §3 (the sum-to-country guarantee that was silently violated) + §6 (now documents both fixes), ADR-023 (the bit-parity mandate the fix preserves), C-58 (the untested-region sibling), C-62 (same file, distinct: joint calibration), resolved C-67/C-69 + open C-70 (the same audit's other findings). | + +The port's `+ 1e-8` denominator epsilon was a float32 **no-op for draw sums ≳ 0.1** (machine epsilon exceeds it — which is why all oracle-parity and gamma-draw tests never saw it) but **silently deflated the scale factor** for tiny nonzero sums: at a draw sum of 1e-8 a country total of 100 reconciled to 50, with no error signal, violating the stated §3 guarantee inside the documented input domain — the audit's one **hard falsification**. A negative country total was likewise neither rejected nor conserved (clamped to zero). **Tier 2** — silent violation of a stated contract guarantee, demonstrated in-domain; not Tier 1 because realistic fatality-scale draws sit well above the band. **Resolved** (2026-07-02): the epsilon is replaced by an explicit all-zero-draw guard (exact division for any nonzero sum; all-zero draws stay zero exactly as before), and negative totals now raise `ValueError`. The fix is **bit-identical on all realistic data** (torch-oracle parity green, unchanged); conservation now exact at float32 precision across the whole domain. Pinned by `tests/test_falsification_safety_audit_2026_07.py` (TestF1HardEpsilonRegion, TestF8SoftNegativeCountryTotal); Reconcile.md §6 updated; `proportional.py` module docstring records the deliberate deviation from the torch original. + +--- + +### C-67: the published conformance suites were a silent no-op under `python -O` — RESOLVED + +| Field | Value | +|-------|-------| +| ID | C-67 | +| Tier | 4 | +| Source | repo-assimilation (2026-07-02) + falsify audit 2026-07-02 (F3 — a float64 non-conformer with working save/load passed the envelope under `-O`) | +| Trigger | (historical) A consumer wiring `assert_frame_contract` / `assert_summarizer_contract` / `assert_reconcile_contract` into a CI or production job running `python -O`/`-OO` — every check silently passed regardless of conformance. | +| Location | `src/views_frames/conformance/__init__.py`, `src/views_frames_summarize/conformance.py`, `src/views_frames_reconcile/conformance.py` (bare `assert` throughout — stripped under optimized bytecode). | +| Cross-refs | ADR-016 (the cross-repo floor whose teeth this restores), C-46 (the "verification depends on how consumers run it" family). | + +The three published conformance suites — the ADR-016 floor consumers run in *their* CI — were bare `assert` statements: under `-O` the interpreter strips them and the suite reports green while checking nothing. **Resolved** (2026-07-02): each suite's public entry points now call `_require_assertions()`, which raises `RuntimeError` when `__debug__` is false — under `-O` the suite **refuses to run loudly** instead of lying. Pinned by a subprocess regression test (`TestF3SoftConformanceUnderO`). **Tier 4** — no consumer was known to run optimized bytecode; latent sharp edge on the verification surface, not a data-corruption path. + +--- + +### C-69: empty-index `searchsorted` crashed with an obscure `IndexError` — RESOLVED + +| Field | Value | +|-------|-------| +| ID | C-69 | +| Tier | 4 | +| Source | repo-assimilation (2026-07-02, the `np.clip(pos, 0, -1)` observation) + falsify audit 2026-07-02 (F4 — confirmed empirically) | +| Trigger | (historical) Any same-level join against an **empty** `SpatioTemporalIndex` (e.g. aligning to a frame filtered down to zero rows) — `searchsorted` clipped positions into an empty array and died with `IndexError: index -1 is out of bounds` instead of a clean result. | +| Location | `src/views_frames/index.py` (`searchsorted`, the `np.clip(pos, 0, len-1)` corner at the formerly-unguarded empty-self path). | +| Cross-refs | C-57 (the same loud-but-obscure error family — `map_estimate`'s inf `IndexError`, still open with #89), C-21 (row-semantics territory). | + +**Resolved** (2026-07-02): an explicit empty-self early-return — every row of `other` is absent from an empty index, so the result is all `-1`, the method's documented not-found value. Pinned by `TestF4SoftEmptyIndexSearchsorted`. **Tier 4** — a crash (loud), not silent corruption; ergonomics-grade. The falsify audit's bonus observation (a *frame* passed to `reindex` surfaces as `AttributeError` rather than a clean `TypeError`) is noted here, deliberately unfixed — same family, below threshold. + +--- + ### C-65: non-finite fail-loud proven only on the single-shot path, not the blocked (multi-block) path — RESOLVED | Field | Value | @@ -961,7 +1036,7 @@ A spatial-forecasting showcase with no spatial display under-serves the audience - **reconcile method + governance** = {C-58, C-62, + D-12; resolved C-64, C-37-lineage} — the per-draw `proportional` reconciler is a pragmatic, information-losing port (C-62) whose principled joint upgrade is deferred (ADR-024); its cutover-verification residual (C-58) is the remaining governance debt, with the mode-reporting decision recorded as D-12. The package's **missing CIC** (C-64) was the other half of the governance debt — closed by `docs/CICs/Reconcile.md` (epic #179 / S1). - **construction-convenience accretion (#113)** = {C-52, C-53, C-54, + D-09} — the planned `PredictionFrame.from_arrays` factory is the "camel's nose" for leaf bloat: accretion (C-52), two frozen construction paths diverging (C-53), and a DoD that overstates scope (C-54). Gated on the #113 decision (D-09). - **cross-repo coordination** = {C-13, C-46, D-04, D-05, D-06} — an N-consumer leaf whose buy-in is *assumed, not elicited*: the concentration/fan-out risk (C-13), the envelope re-assertion in views-evaluation (C-46), plus the unratified-perspective disagreements. Resolvable only across repos, not within the leaf. - - **immutability enforcement** = {resolved C-63, C-07} — **resolved by ADR-025 (2026-06-28, epic #179 / S2).** Immutability is enforced for the *index* (`setflags(write=False)`) and held *by convention* for the *value buffer* (writeable on purpose, to preserve zero-copy / `mmap`; mutating `.values` in place is unsupported). The contract was corrected to this reality across the three frame CICs + README design principle 3; the `setflags`-enforce on `.values` would be a MAJOR ("tightening an invariant", GOVERNANCE/ADR-018) and is recorded as a deferred MAJOR-rider. + - **immutability enforcement** = {C-66, + resolved C-63, C-07} — the **contract-correction** half is done (**C-63 resolved** by ADR-025, 2026-06-28, epic #179 / S2): immutability is enforced for the *index* (`setflags(write=False)`) and held *by convention* for the *value buffer* (writeable on purpose, to preserve zero-copy / `mmap`; mutating `.values` is documented-unsupported across the three frame CICs + README design principle 3). The **enforcement** half — `setflags(write=False)` on `.values` — is a MAJOR ("tightening an invariant", GOVERNANCE/ADR-018) and is **deferred, tracked open as C-66** (the enforce-rider for the next MAJOR), so the residual writeable-buffer exposure stays visible rather than buried in the resolved C-63. - cross-cutting **verification-completeness** = {C-58, resolved C-65} — a guard or path is structurally correct but not adversarially pinned: the reconciler's production-slice check (C-58, still open). Its sibling — the non-finite fail-loud on the blocked/multi-block path (C-65) — was **resolved by a red test (2026-06-28, epic #179 / S3)** placing a non-finite draw in a non-first block via `block_rows`. - **post-1.1.0 polish** = {C-35, C-36, C-37, C-38} — **resolved by Epic 7 (2026-06-24)**. Low-severity doc/test-completeness items from the 2026-06-24 repo-assimilation + test-review; closed before the v1.1.0 `main` merge, no `src/` behaviour change. - **test-coverage debt** = {C-29, C-31} — **resolved by Epic 6 (2026-06-23)**. Fail-loud / parity paths that existed in code but lacked tests (root cause: the v1.0.0 suite optimized happy-path coverage over failure/parity branches); now closed with a CI 100%-coverage gate. diff --git a/src/views_frames/conformance/__init__.py b/src/views_frames/conformance/__init__.py index c93a748..f58b0b9 100644 --- a/src/views_frames/conformance/__init__.py +++ b/src/views_frames/conformance/__init__.py @@ -40,6 +40,20 @@ ] +def _require_assertions() -> None: + """Fail loud if assertions are stripped (``python -O``/``-OO``). + + The suite is built from ``assert`` statements; under optimized bytecode every + check would silently pass regardless of conformance — a verification tool that + reports green while dead (falsify audit 2026-07, F3). Refuse to run instead. + """ + if not __debug__: # pragma: no cover — pytest always runs with assertions on + raise RuntimeError( + "the views-frames conformance suite requires assertions; run without " + "python -O/-OO (PYTHONOPTIMIZE), otherwise every check silently passes" + ) + + def assert_frame_envelope(frame: Any) -> None: """Assert ``frame`` satisfies the shared **frame envelope**. @@ -56,6 +70,7 @@ def assert_frame_envelope(frame: Any) -> None: Raises: AssertionError: any part of the envelope is violated. """ + _require_assertions() values = frame.values assert isinstance(values, np.ndarray), "values must be a numpy array" assert values.dtype == np.float32, f"values must be float32, got {values.dtype}" @@ -77,6 +92,7 @@ def assert_frame_contract(frame: Any) -> None: Raises: AssertionError: the envelope or the spatiotemporal identifier rule is violated. """ + _require_assertions() assert_frame_envelope(frame) ids = frame.identifiers @@ -110,6 +126,7 @@ def assert_index_alignment_laws(index_a: Any, index_b: Any) -> None: Raises: AssertionError: a law is violated. """ + _require_assertions() assert index_a.intersect(index_b) == index_b.intersect(index_a), ( "intersect must be commutative" ) @@ -140,6 +157,7 @@ def assert_cross_level_alignment_law( AssertionError: the remap disagrees with the mapping, drops time, or produces the wrong level. """ + _require_assertions() aligned = index.cross_level_align(mapping, target_level) assert aligned.level is target_level, "cross_level_align must carry target_level" assert np.array_equal(aligned.time, index.time), "cross_level_align must keep time" diff --git a/src/views_frames/index.py b/src/views_frames/index.py index eca6e21..a2871d9 100644 --- a/src/views_frames/index.py +++ b/src/views_frames/index.py @@ -132,6 +132,11 @@ def searchsorted(self, other: SpatioTemporalIndex) -> NDArray[np.intp]: The pure-numpy analogue of ``pd.Index.get_indexer``: a same-level join. """ self._require_same_level(other) + if self.n_rows == 0: + # An empty index contains nothing: every row of `other` is absent. The + # general path below would clip positions to -1 and crash with an + # obscure IndexError on the empty array (falsify audit 2026-07, F4). + return np.full(other.n_rows, -1, dtype=np.intp) self_rows = self._row_view(self._keys()) other_rows = self._row_view(other._keys()) order = np.argsort(self_rows, kind="stable") diff --git a/src/views_frames_reconcile/conformance.py b/src/views_frames_reconcile/conformance.py index 437eb40..261eef1 100644 --- a/src/views_frames_reconcile/conformance.py +++ b/src/views_frames_reconcile/conformance.py @@ -20,6 +20,19 @@ from views_frames_reconcile.module import ReconciliationModule +def _require_assertions() -> None: + """Fail loud if assertions are stripped (``python -O``/``-OO``). + + Mirrors ``views_frames.conformance._require_assertions`` (falsify audit 2026-07, + F3): under optimized bytecode every ``assert`` silently passes — refuse to run. + """ + if not __debug__: # pragma: no cover — pytest always runs with assertions on + raise RuntimeError( + "the reconcile conformance suite requires assertions; run without " + "python -O/-OO (PYTHONOPTIMIZE), otherwise every check silently passes" + ) + + def assert_reconcile_contract( cm_frame: PredictionFrame, pgm_frame: PredictionFrame, @@ -35,6 +48,7 @@ def assert_reconcile_contract( Raises: AssertionError: a contract law is violated. """ + _require_assertions() mk = np.asarray(map_keys) mv = np.asarray(map_vals) out = ReconciliationModule(mk, mv).reconcile(cm_frame, pgm_frame) diff --git a/src/views_frames_reconcile/proportional.py b/src/views_frames_reconcile/proportional.py index e3174cd..beb5c16 100644 --- a/src/views_frames_reconcile/proportional.py +++ b/src/views_frames_reconcile/proportional.py @@ -9,6 +9,11 @@ This is a *faithful, numpy-only* port of views-reporting's ``ForecastReconciler.reconcile_forecast`` (torch), migrated here because the algorithm belongs in post-processing, not reporting (views-reporting issue #72). +One deliberate deviation (falsify audit 2026-07): the original's ``+ 1e-8`` +denominator epsilon is replaced by an explicit all-zero-draw guard — bit-identical +on all realistic data (the epsilon was a float32 no-op for draw sums ≳ 0.1) but +exactly conserving for tiny nonzero sums, where the epsilon silently deflated the +scale factor. Negative country totals now fail loud instead of silently clamping. It is intentionally the **same** method — a pragmatic per-draw approximation, not principled joint probabilistic reconciliation. The upgrade to the latter is designed in **ADR-024** (register **C-62**; the cross-repo lineage is views-postprocessing @@ -27,8 +32,6 @@ import numpy as np from numpy.typing import NDArray -_EPS = np.float32(1e-8) - def reconcile_proportional( grid: NDArray[np.floating[Any]] | object, @@ -49,7 +52,8 @@ def reconcile_proportional( are clamped to be non-negative. Raises: - ValueError: the grid and country sample counts disagree. + ValueError: the grid and country sample counts disagree, or any country + total is negative (cannot be conserved under the non-negativity clamp). """ grid_arr = np.asarray(grid, dtype=np.float32) is_point = grid_arr.ndim == 1 @@ -65,13 +69,32 @@ def reconcile_proportional( f"Mismatch in sample count: grid has {grid_arr.shape[0]}, " f"country has {country_arr.shape[0]}" ) + if bool((country_arr < 0).any()): + # A negative total cannot be conserved under the non-negativity clamp — the + # output would silently sum to 0, not the total (falsify audit 2026-07, F8). + raise ValueError( + "country totals must be non-negative; got a negative total " + f"(min={float(country_arr.min())}). A negative forecast is an upstream " + "bug; reconciliation cannot conserve it under the non-negativity clamp." + ) # Preserve zeros: only strictly-positive cells carry probability mass. nonzero = np.where(grid_arr > 0, grid_arr, np.float32(0.0)) - # Per-draw proportional scaling to the (authoritative) country total. + # Per-draw proportional scaling to the (authoritative) country total. Guard the + # all-zero draws explicitly (they carry no mass and stay zero — the documented + # edge) instead of an additive epsilon in the denominator: `+ 1e-8` was a no-op + # in float32 for sums ≳ 0.1 (machine eps exceeds it) but silently deflated the + # scale factor for tiny nonzero sums — at a draw sum of 1e-8 a country total of + # 100 reconciled to 50 with no error signal (falsify audit 2026-07, F1). The + # exact division below is bit-identical to the epsilon form on all realistic + # data (torch-oracle parity preserved) and conserves exactly for any nonzero sum. sum_nonzero = nonzero.sum(axis=1, keepdims=True) # (S, 1) - scaling = country_arr.reshape(-1, 1) / (sum_nonzero + _EPS) # (S, 1) + has_mass = sum_nonzero > 0 + safe_sum = np.where(has_mass, sum_nonzero, np.float32(1.0)) + scaling = np.where( + has_mass, country_arr.reshape(-1, 1) / safe_sum, np.float32(0.0) + ) # (S, 1) adjusted = np.clip(nonzero * scaling, 0.0, None).astype(np.float32) return np.asarray(adjusted[0] if is_point else adjusted, dtype=np.float32) diff --git a/src/views_frames_summarize/conformance.py b/src/views_frames_summarize/conformance.py index be48f44..2bfe456 100644 --- a/src/views_frames_summarize/conformance.py +++ b/src/views_frames_summarize/conformance.py @@ -22,12 +22,26 @@ from views_frames_summarize.tower_point import tower_point +def _require_assertions() -> None: + """Fail loud if assertions are stripped (``python -O``/``-OO``). + + Mirrors ``views_frames.conformance._require_assertions`` (falsify audit 2026-07, + F3): under optimized bytecode every ``assert`` silently passes — refuse to run. + """ + if not __debug__: # pragma: no cover — pytest always runs with assertions on + raise RuntimeError( + "the summarize conformance suite requires assertions; run without " + "python -O/-OO (PYTHONOPTIMIZE), otherwise every check silently passes" + ) + + def assert_summarizer_contract(frame: AnyFrame) -> None: """Assert the summarizers behave on ``frame``. Raises: AssertionError: a summarizer violates its output contract. """ + _require_assertions() n = frame.n_rows point = collapse(frame, np.mean) diff --git a/tests/test_falsification_safety_audit_2026_07.py b/tests/test_falsification_safety_audit_2026_07.py new file mode 100644 index 0000000..e84bea4 --- /dev/null +++ b/tests/test_falsification_safety_audit_2026_07.py @@ -0,0 +1,129 @@ +"""Regression pins from the falsification audit 2026-07-02 (four-axis audit, Phase 4). + +Each test pins a falsification of the safety claim found (and since fixed) by the +audit — written first as failing stubs, turned green by the fixes: + + F1 (HARD) — reconcile silently violated sum-to-country for tiny nonzero draw sums + (the `+ 1e-8` denominator epsilon; replaced by an explicit all-zero guard). + F3 (soft) — the published conformance suites were a silent no-op under `python -O` + (bare asserts; now guarded by `_require_assertions`, which refuses to run). + F4 (soft) — empty-index `searchsorted` died with an obscure IndexError + (the `np.clip(pos, 0, -1)` corner; now returns all -1, the not-found value). + F8 (soft) — a negative country total silently clamped to zero, breaking + conservation (now raises ValueError). +""" + +from __future__ import annotations + +import subprocess +import sys +import textwrap + +import numpy as np + +from views_frames import PredictionFrame, SpatialLevel, SpatioTemporalIndex + + +def _pair(grid_draws): + """A 2-cell pgm frame + 1-country cm frame (total 100) + mapping.""" + t = np.array([1, 1], dtype=np.int64) + u = np.array([100, 101], dtype=np.int64) + mk = np.array([[1, 100], [1, 101]], dtype=np.int64) + mv = np.array([7, 7], dtype=np.int64) + pg_idx = SpatioTemporalIndex(t, u, SpatialLevel.PGM) + cm_idx = SpatioTemporalIndex( + np.array([1], dtype=np.int64), np.array([7], dtype=np.int64), SpatialLevel.CM + ) + pgm = PredictionFrame(np.asarray(grid_draws, dtype=np.float32), pg_idx) + cm = PredictionFrame(np.array([[100.0]], dtype=np.float32), cm_idx) + return cm, pgm, mk, mv + + +class TestF1HardEpsilonRegion: + """F1 (HARD): tiny-but-nonzero draws are 'active' per the conformance law's own + definition (in_sum != 0), yet reconcile's `sum_nonzero + _EPS` denominator + (formerly proportional.py:74) silently deflated the scale factor — at grid-sum + 1e-8 the reconciled sum was 50 instead of 100. Fixed: exact division behind an + explicit all-zero guard; conservation now holds for any nonzero sum.""" + + def test_own_conformance_law_holds_on_tiny_nonzero_draws(self): + from views_frames_reconcile.conformance import assert_reconcile_contract + + cm, pgm, mk, mv = _pair([[6e-9], [4e-9]]) + # Green post-fix: the sum-to-country law holds on tiny nonzero draws. + assert_reconcile_contract(cm, pgm, mk, mv) + + def test_conservation_relative_error_within_documented_tolerance(self): + from views_frames_reconcile import ReconciliationModule + + cm, pgm, mk, mv = _pair([[6e-6], [4e-6]]) # grid sum 1e-5 — still violating + out = ReconciliationModule(mk, mv).reconcile(cm, pgm) + rel = abs(float(out.values.sum()) - 100.0) / 100.0 + # Green post-fix: exact division conserves within the §3 rtol. + assert rel <= 1e-4, f"silent conservation violation: rel_err={rel:.4%}" + + +class TestF3SoftConformanceUnderO: + """F3 (soft): the published conformance suites are bare `assert`s — under + `python -O` a float64 non-conformer with working save/load PASSED the envelope. + Fixed: `_require_assertions()` makes the suite refuse to run under -O (exit 1 + here comes from that RuntimeError — the point is it can no longer silently pass).""" + + def test_envelope_rejects_nonconformer_even_under_optimized_bytecode(self): + code = textwrap.dedent(""" + import numpy as np, pathlib, sys + class Subtle: + def __init__(self, values): + self.values = values; self.n_rows = values.shape[0] + self.identifiers = {"time": np.array([1,2]), "unit": np.array([10,20])} + def save(self, d): np.save(pathlib.Path(d)/"v.npy", self.values) + @classmethod + def load(cls, d, mmap=False): return cls(np.load(pathlib.Path(d)/"v.npy")) + from views_frames.conformance import assert_frame_envelope + try: + assert_frame_envelope(Subtle(np.ones((2,1), dtype=np.float64))) + sys.exit(0) # passed: the envelope had no teeth + except AssertionError: + sys.exit(1) # rejected: teeth present + """) + r = subprocess.run([sys.executable, "-O", "-c", code], capture_output=True) + # exit 1 = rejected OR refused-to-run; exit 0 would mean a silent pass. + assert r.returncode == 1, "conformance envelope is a no-op under python -O" + + +class TestF4SoftEmptyIndexSearchsorted: + """F4 (soft): searchsorted from an EMPTY index against a non-empty one raises an + obscure IndexError (the np.clip(pos, 0, -1) corner, index.py:140) instead of a + clean result/-1s or a named ValueError. Fixed: returns all -1 (not-found).""" + + def test_empty_index_searchsorted_is_clean(self): + empty = SpatioTemporalIndex( + np.array([], dtype=np.int64), np.array([], dtype=np.int64), SpatialLevel.PGM + ) + full = SpatioTemporalIndex( + np.array([1, 2], dtype=np.int64), np.array([10, 20], dtype=np.int64), + SpatialLevel.PGM, + ) + # Green post-fix: all -1 (the not-found value), matching searchsorted's + # documented missing-row semantics. + try: + pos = empty.searchsorted(full) + except ValueError: + return # a clean, documented raise is acceptable + assert (np.asarray(pos) == -1).all() + + +class TestF8SoftNegativeCountryTotal: + """F8 (soft): a negative country total is neither rejected nor conserved — the + non-negativity clamp silently mapped everything to 0. Fixed: raises ValueError.""" + + def test_negative_country_total_rejected_or_conserved(self): + from views_frames_reconcile.proportional import reconcile_proportional + + grid = np.array([3.0, 7.0], dtype=np.float32) + # Green post-fix: raises ValueError (upstream nonsense rejected loudly). + try: + out = reconcile_proportional(grid, -50.0) + except ValueError: + return + assert np.isclose(float(np.asarray(out).sum()), -50.0) diff --git a/tests/test_io.py b/tests/test_io.py index c150d24..9dcdf93 100644 --- a/tests/test_io.py +++ b/tests/test_io.py @@ -59,6 +59,18 @@ def test_npz_mmap_returns_memmap(tmp_path): assert isinstance(out["values"], np.memmap) +def test_npz_mmap_is_read_only(tmp_path): + # The mmap load opens with mmap_mode="r": the buffer must be non-writeable and + # an in-place write must raise — previously asserted only as isinstance(memmap) + # (2026-07 audit, F2 / register C-70; the read-only-ness is the safety property). + st = _state_2d() + npz.save(tmp_path, **st) + out = npz.load(tmp_path, mmap=True) + assert out["values"].flags.writeable is False + with pytest.raises(ValueError, match="read-only"): + out["values"][0, 0] = 99.0 + + def test_npz_builds_index_from_state(tmp_path): st = _state_2d() npz.save(tmp_path, **st) diff --git a/tests/test_reconciliation_parity.py b/tests/test_reconciliation_parity.py index fdf48e9..e5d83cb 100644 --- a/tests/test_reconciliation_parity.py +++ b/tests/test_reconciliation_parity.py @@ -78,3 +78,18 @@ def test_sample_count_mismatch_raises(self): country = np.zeros(200, dtype=np.float32) with pytest.raises(ValueError, match="Mismatch in sample count"): reconcile_proportional(grid, country) + + def test_shares_are_preserved_within_each_draw(self): + # The method's DEFINING property — top-down *forecast-proportion* + # disaggregation: within a draw, every cell keeps its relative share of + # the nonzero mass. Until now this was pinned only by the frozen-oracle + # fixtures (an equal-split-among-nonzero mutation would have passed every + # law test); the 2026-07 audit (F5) proved the law holds — commit it as a + # law so the essence is oracle-independent (register C-70). + rng = np.random.default_rng(3) + grid = rng.gamma(2.0, 5.0, size=(50, 8)).astype(np.float32) + country = grid.sum(axis=1) * np.float32(1.7) + adjusted = reconcile_proportional(grid, country) + in_shares = grid / grid.sum(axis=1, keepdims=True) + out_shares = adjusted / adjusted.sum(axis=1, keepdims=True) + np.testing.assert_allclose(out_shares, in_shares, atol=1e-6) diff --git a/tests/test_reconciliation_validation.py b/tests/test_reconciliation_validation.py index 46ea2a5..66a8982 100644 --- a/tests/test_reconciliation_validation.py +++ b/tests/test_reconciliation_validation.py @@ -85,3 +85,12 @@ def test_missing_country_raises(self, fix): vals=fix["cm__pred_ged_sb"][keep]) with pytest.raises(ValueError, match="no country forecast"): validate_reconciliation_inputs(cm, _pgm(fix), mk, mv) + + def test_missing_mapping_entry_raises(self, fix): + # Reconcile.md §6: a grid row whose (time, priogrid_gid) is absent from the + # injected mapping fails loud (via cross_level_align). Every other "missing" + # test drops a cm row; this one drops a MAP KEY — previously pinned only at + # the leaf, never from the reconcile suite (2026-07 audit / register C-70). + mk, mv = _mk(fix) + with pytest.raises(ValueError, match="no entry in the injected mapping"): + validate_reconciliation_inputs(_cm(fix), _pgm(fix), mk[:-1], mv[:-1]) diff --git a/uv.lock b/uv.lock index 106d0ed..bde9d42 100644 --- a/uv.lock +++ b/uv.lock @@ -3274,7 +3274,7 @@ wheels = [ [[package]] name = "views-frames" -version = "1.8.0" +version = "1.8.1" source = { editable = "." } dependencies = [ { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },