Skip to content

Show the response ID, and assert user text stays out of the log - #32

Merged
Robbie1977 merged 6 commits into
mainfrom
feat/response-id-and-console-guard
Aug 14, 2026
Merged

Show the response ID, and assert user text stays out of the log#32
Robbie1977 merged 6 commits into
mainfrom
feat/response-id-and-console-guard

Conversation

@Robbie1977

@Robbie1977 Robbie1977 commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

The response identifier was never rendered

The DPIA describes a data-subject-rights route in which someone asks about the data held on a particular exchange by quoting its response identifier. The identifier was generated and stored from the start and never shown, so the route could not be exercised and an assessor could disprove it in a browser in under a minute.

It now appears under every answer, not only the latest — the feedback prompt is deliberately rendered once at the foot of the conversation, but the answer someone wants to ask about is often not the last one. Quiet: 0.68em, muted, below the sources line, selectable, with a copy button that degrades to "select and copy" where the clipboard API is unavailable. The privacy notice says what the ID is for, since we hold no account and no name and it is the only way to find the records for one exchange.

Nothing asserted that A1b stayed closed

safeToolArgs renders tool arguments by shape rather than by value, and nothing tested it. The surface had already been widened three times by diagnostics added for good reasons, each caught by somebody reading the code rather than by a failing test.

The new test drives the real paths — runHarness with mocked ELM and MCP, the absence escalation, the absence gate, the tool-failure reporter — with a canary planted where the user's text enters, and asserts it never reaches the console. Being behavioural rather than a grep, a new diagnostic that interpolates user text fails it without anyone remembering the file exists.

It caught four live leaks, all in orchestrator.mjs, none of them tool arguments — which is why safeToolArgs did not cover them:

  • the requested term name in get_term_info FAILED
  • the requested term name in deprecated term redirected
  • the verbatim absence sentence in ABSENCE ESCALATION
  • the removed sentence in ABSENCE GATE

A term name is often the user's phrase, and an absence sentence is prose the model wrote by restating the question — the model writes a denial by restating what was asked, so the wording comes back out in it.

The two term names go through safeText, a sibling of safeToolArgs with the same VFB_HARNESS_TRACE escape hatch: term=<text:27>. A term name has no shape worth reporting, and the line already carries the ontology id, which says which lookup failed far more precisely than the name did.

The two absence lines get something better than a length. Each entry in ABSENCE_PATTERNS now carries a short stable slug and findAbsenceClaims reports which one matched, so the log reads:

[VFBchat] ABSENCE ESCALATION | queries=SubClasses@FBbt_00005095,… | claimed=absence:does-not-hold | len=63
[VFBchat] ABSENCE GATE | removed=1 | licensed=false | absence:does-not-hold

That is strictly more useful than the sentence was — the shape of denial is countable across runs, so a rise in no-data-in becomes something you can see — and it carries none of the user's wording. repairUnlicensedAbsences returns the slugs alongside the strings rather than instead of them; the strings still reach the per-request trace, which is off unless VFB_HARNESS_TRACE is set.

Net effect: the ids, counts and query types that make these diagnostics useful are unchanged, and no free text reaches the container log on any path.

How to test

npm test (8 new cases in tests/unit/consoleLeakGuard.test.mjs). The WCAG job covers the new button; check the copy control by keyboard on the deployed build.

Robbie1977 and others added 6 commits August 13, 2026 12:35
The DPIA describes a data-subject-rights route in which someone asks about
the data held on a particular exchange by quoting its response identifier.
The identifier was generated and stored from the start and never rendered,
so the route could not be exercised and an assessor could disprove it in a
browser in under a minute.

On every answer, not only the latest. The feedback prompt is deliberately
rendered once at the foot of the conversation, but the answer someone wants
to ask about is often not the last one.

Quiet: 0.68em, muted, below the sources line, selectable, with a copy button
that degrades to "select and copy" where the clipboard API is unavailable
(plain HTTP, some embedded browsers). The privacy notice now says what the
ID is for, since we hold no account and no name and it is the only way to
find the records for one exchange.
safeToolArgs closed finding A1b by rendering tool arguments by shape rather
than by value, and nothing asserted it stayed closed. The surface had
already been widened three times by diagnostics added for good reasons,
each caught by somebody reading the code rather than by a failing test.

The new test drives the real paths — runHarness with mocked ELM and MCP,
the absence escalation, the absence gate, the tool-failure reporter — with
a canary planted where the user's text enters, and asserts it never reaches
the console. Being behavioural rather than a grep, a new diagnostic that
interpolates user text fails it without anyone remembering the file exists.

It caught four live leaks, all in orchestrator.mjs and none of them tool
arguments, which is why safeToolArgs did not cover them: the requested term
name in the term-info failure report and in the deprecated-term redirect,
and the verbatim absence sentence in the escalation and gate lines. A term
name is often the user's phrase, and an absence sentence is prose the model
wrote by restating the question. All four now go through safeText, a
sibling of safeToolArgs with the same VFB_HARNESS_TRACE escape hatch, so
the ids and counts that make the diagnostics useful are unchanged and only
the free text is rendered as a length.
The previous commit rendered the escalation and gate lines' absence
sentence as a length. That closed the leak and left nothing behind, and
those two lines exist to make the rate at which answers try to deny data
visible in the deployment.

Each entry in ABSENCE_PATTERNS now carries a short stable slug, and
findAbsenceClaims reports which one matched. The log says
claimed=absence:does-not-hold rather than a length or a sentence, which is
strictly more useful than the sentence was: the shape of denial is
countable across runs, and it contains none of the user's wording.

repairUnlicensedAbsences returns the slugs alongside the strings rather
than instead of them — the strings still go to the per-request trace, which
is off unless VFB_HARNESS_TRACE is set. The two term-name lines keep the
length rendering: a term name has no shape worth reporting.
lib/orchestrator.mjs: #30 added discardDraft() immediately after the ABSENCE
ESCALATION log line that this branch rewrites. Both are kept — the line no
longer quotes the user's sentence, and the streamed draft is still withdrawn.
latest.json is generated and takes this branch's run.

# Conflicts:
#	lib/orchestrator.mjs
#	test-results/task-battery/latest.json
@Robbie1977
Robbie1977 merged commit 6b4872a into main Aug 14, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants