Show the response ID, and assert user text stays out of the log - #32
Merged
Conversation
The DPIA describes a data-subject-rights route in which someone asks about the data held on a particular exchange by quoting its response identifier. The identifier was generated and stored from the start and never rendered, so the route could not be exercised and an assessor could disprove it in a browser in under a minute. On every answer, not only the latest. The feedback prompt is deliberately rendered once at the foot of the conversation, but the answer someone wants to ask about is often not the last one. Quiet: 0.68em, muted, below the sources line, selectable, with a copy button that degrades to "select and copy" where the clipboard API is unavailable (plain HTTP, some embedded browsers). The privacy notice now says what the ID is for, since we hold no account and no name and it is the only way to find the records for one exchange.
safeToolArgs closed finding A1b by rendering tool arguments by shape rather than by value, and nothing asserted it stayed closed. The surface had already been widened three times by diagnostics added for good reasons, each caught by somebody reading the code rather than by a failing test. The new test drives the real paths — runHarness with mocked ELM and MCP, the absence escalation, the absence gate, the tool-failure reporter — with a canary planted where the user's text enters, and asserts it never reaches the console. Being behavioural rather than a grep, a new diagnostic that interpolates user text fails it without anyone remembering the file exists. It caught four live leaks, all in orchestrator.mjs and none of them tool arguments, which is why safeToolArgs did not cover them: the requested term name in the term-info failure report and in the deprecated-term redirect, and the verbatim absence sentence in the escalation and gate lines. A term name is often the user's phrase, and an absence sentence is prose the model wrote by restating the question. All four now go through safeText, a sibling of safeToolArgs with the same VFB_HARNESS_TRACE escape hatch, so the ids and counts that make the diagnostics useful are unchanged and only the free text is rendered as a length.
The previous commit rendered the escalation and gate lines' absence sentence as a length. That closed the leak and left nothing behind, and those two lines exist to make the rate at which answers try to deny data visible in the deployment. Each entry in ABSENCE_PATTERNS now carries a short stable slug, and findAbsenceClaims reports which one matched. The log says claimed=absence:does-not-hold rather than a length or a sentence, which is strictly more useful than the sentence was: the shape of denial is countable across runs, and it contains none of the user's wording. repairUnlicensedAbsences returns the slugs alongside the strings rather than instead of them — the strings still go to the per-request trace, which is off unless VFB_HARNESS_TRACE is set. The two term-name lines keep the length rendering: a term name has no shape worth reporting.
lib/orchestrator.mjs: #30 added discardDraft() immediately after the ABSENCE ESCALATION log line that this branch rewrites. Both are kept — the line no longer quotes the user's sentence, and the streamed draft is still withdrawn. latest.json is generated and takes this branch's run. # Conflicts: # lib/orchestrator.mjs # test-results/task-battery/latest.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The response identifier was never rendered
The DPIA describes a data-subject-rights route in which someone asks about the data held on a particular exchange by quoting its response identifier. The identifier was generated and stored from the start and never shown, so the route could not be exercised and an assessor could disprove it in a browser in under a minute.
It now appears under every answer, not only the latest — the feedback prompt is deliberately rendered once at the foot of the conversation, but the answer someone wants to ask about is often not the last one. Quiet: 0.68em, muted, below the sources line, selectable, with a copy button that degrades to "select and copy" where the clipboard API is unavailable. The privacy notice says what the ID is for, since we hold no account and no name and it is the only way to find the records for one exchange.
Nothing asserted that A1b stayed closed
safeToolArgsrenders tool arguments by shape rather than by value, and nothing tested it. The surface had already been widened three times by diagnostics added for good reasons, each caught by somebody reading the code rather than by a failing test.The new test drives the real paths —
runHarnesswith mocked ELM and MCP, the absence escalation, the absence gate, the tool-failure reporter — with a canary planted where the user's text enters, and asserts it never reaches the console. Being behavioural rather than a grep, a new diagnostic that interpolates user text fails it without anyone remembering the file exists.It caught four live leaks, all in
orchestrator.mjs, none of them tool arguments — which is whysafeToolArgsdid not cover them:get_term_info FAILEDdeprecated term redirectedABSENCE ESCALATIONABSENCE GATEA term name is often the user's phrase, and an absence sentence is prose the model wrote by restating the question — the model writes a denial by restating what was asked, so the wording comes back out in it.
The two term names go through
safeText, a sibling ofsafeToolArgswith the sameVFB_HARNESS_TRACEescape hatch:term=<text:27>. A term name has no shape worth reporting, and the line already carries the ontology id, which says which lookup failed far more precisely than the name did.The two absence lines get something better than a length. Each entry in
ABSENCE_PATTERNSnow carries a short stable slug andfindAbsenceClaimsreports which one matched, so the log reads:That is strictly more useful than the sentence was — the shape of denial is countable across runs, so a rise in
no-data-inbecomes something you can see — and it carries none of the user's wording.repairUnlicensedAbsencesreturns the slugs alongside the strings rather than instead of them; the strings still reach the per-request trace, which is off unlessVFB_HARNESS_TRACEis set.Net effect: the ids, counts and query types that make these diagnostics useful are unchanged, and no free text reaches the container log on any path.
How to test
npm test(8 new cases intests/unit/consoleLeakGuard.test.mjs). The WCAG job covers the new button; check the copy control by keyboard on the deployed build.