Feat/0.8.0 - #16
Merged
Merged
Conversation
The last message of the 0.7.0 `qwen3.5:4b` evaluation session was "It all works
now, thank you". The agent answered it by building a checklist and starting to
re-fix bugs it had already fixed — items carried over from two turns earlier.
The user cancelled the run.
Traced through `classify`, the message matches nothing: no mutating verb, no work
verb, no file, and `isPurelySocial` rejects it because `works` and `now` are not
social words. It reached the `task` default.
The gap is a category, not two words. Adding `works` to the social vocabulary
would be wrong — "the delete button no longer works" is a bug report. So a
success report is matched as a phrase, and any sign the sentence goes on to say
something is still wrong (`but`, `still`, a negation, `almost`) hands it straight
back to the agent. It is checked after the mutating-verb rule, so "it works now,
can you also add a dark mode" stays a task.
Fixing that surfaced a live bug in `isGreetingWithName`, which tested the first
word against the whole of SOCIAL_WORDS. That set is mostly filler — "it", "the",
"got", "all" — admitted there on the strength of a rule that only holds for whole
messages. Read one word at a time it made any message of three words or fewer a
greeting, so on main today:
"it doesn't work" -> chat
"the tests fail" -> chat
"got an error" -> chat
"all buttons broken" -> chat
Four bug reports answered conversationally with the request dropped, which is the
one outcome this module's header says it must never produce. Greetings now match
against a dedicated GREETING_WORDS set.
The new pattern is flagged `detect-unsafe-regex`; it was measured rather than
argued. Every optional group carries a distinct literal prefix, so none can claim
the same characters — 60,000-character adversarial inputs run in 0.5ms, flat.
Also adds doc/SESSION-ANALYSIS-0.7.0.md, the counted analysis of both evaluation
sessions that the rest of 0.8.0 follows from.
…s over a listing Both 0.7.0 evaluation sessions lost most of their value to the same missing thing, at opposite ends of the model range. `qwen3.5:0.8b` never wrote a single file. Five of its seven sessions ended `repeating`, four at exactly two steps: list_files, list_files, list_files, run over. Twelve of its twenty-two total steps were list_files. `qwen3.5:4b` finished and took 88 minutes, 97% of it inside the model. 73 of its 126 steps were read_file against 21 writes; the audit log has 263 read entries across 25 distinct paths. App.jsx was read 28 times and written 4. A binary PNG was read into the prompt 13 times. The existing hints fired correctly every time — "You now know what is in the project", "Do NOT do it again" — and both models did it again anyway. That is the finding: every anti-repetition device in the loop is a sentence addressed to the model's judgement, evaluated against a context that no longer contains the thing it describes. The model is asked to take the loop's word for what it already has, and reaching for the tool is the cheaper way to be sure. So `agent/workingSet` keeps the record instead of asserting it — paths read, written, listed, deleted, commands run, and what last went wrong — and renders it back as a standing block. It runs off the step trace, so a 0.8B model gets the same footing `stepBrief` gives a 4B one without needing the 2B checklist threshold it is excluded from. Tier B additionally stops treating a repeated read-only action as fatal. A repeated list_files is five milliseconds of directory read, and the response was to end the user's whole run, while a wrong `npm install` gets a diagnosis and another go. It now gets one substitution: the result it already had, handed back with an instruction naming the next move. Repeat again and the guard ends the run as before. Mutating and executing actions are untouched — repeating run_script can install packages and start servers. On Tier A the block is advisory and is moved rather than appended, so one copy exists and it is always the current one, always last. One existing test changed: Plan mode's "no checklist from the loop" case scripted exactly three identical reads to reach the guard. The third is now answered, so it takes a fourth to arrive at the same stop the test is about.
The trace showed `read_file src/App.jsx ok` — the tool identifier the model is required to emit, the path, and an outcome. Two things were wrong with that. It was collapsed always. The 0.7.0 sessions measured 42 seconds per step and 88 minutes for one task, so the panel sat silent for minutes at a time and the user's first sight of a run going wrong was the summary at the end. The steps are the only evidence available while interrupting is still worth doing. It now opens on the first step and collapses when the turn ends — unless the user has clicked it themselves, after which we stop deciding for them. And it never showed why. Each row now carries the model's own stated reason for the step, which both loops already captured as `thought` and nothing rendered. Without it, eight reads of one file look exactly like eight reads of eight. `read_file` is also gone from the panel in favour of "Reading". The identifier is part of the tool protocol and does not belong in the surface whose job is to explain the run. The row logic is a pure `describeStep` so it is testable — the webview's DOM assembly is not reachable from the unit suite, and adding jsdom to a privacy-first extension to render one row is a poor trade. Two details worth naming. The `toggle` event is the obvious hook for "did the user open this" and the wrong one: it also fires on a programmatic `open`, and fires asynchronously, so a flag set around the assignment is not reliably cleared before it arrives. A click on the summary is unambiguous. And the verb lookup is a Map, because its key is model output — a plain object lookup reaches the prototype, so an action named `constructor` returns `Function`, survives the `||` fallback, and renders a function's source as the name of a step.
…note The hero offered "Install from the VS Code Marketplace". The extension is not on the Marketplace, so the button in the picture was the one thing in it that was not true. It now reads "Download from GitHub Releases" and carries the pre-release note, and the README opens with the same, pointing at the step that already documented the real install. Two new images, both built from the same 1280x720 HTML sources as the existing pair: - live-session.png — the step panel mid-run, six steps of a TODO build with the action, the file, and the model's stated reason on each row. - knows-what-it-has.png — the 0.8B before-and-after, using the measured numbers: three identical list_files and the run ends, 5 of 7 sessions, 0 files written, against a repeat that is answered and a third step that writes. The "New in" tag moves from "Big requests become a checklist" to "Agentic on every model", which is the card 0.8.0 actually changed. The regeneration instructions gained the two Windows failure modes that cost time here, both of which fail silently — Chromium reports success and writes nothing. The page has to be a percent-encoded file:/// URL, and --screenshot will not write to a path containing a space, which every path in this repo does.
The 0.8.0 pass corrected the hero's CTA button but left the same false claim in
the capabilities footer, which read
Search "HirayaCoder" in the Extensions view
That is an instruction that cannot work: the extension is not on the Marketplace,
so searching the Extensions view finds nothing. It now says pre-release and gives
the actual destination.
"Pre-release" is kept alongside the URL rather than the URL alone, because
dropping the line silently leaves a reader wondering why they cannot just search
for it — the absence is worth explaining, not just correcting. The bare host is
used in preference to a full https:// URL: nothing in a PNG is clickable, so the
scheme is noise.
The regeneration notes gain the install route as a thing to review per release,
naming both files that carry it, so the day this does reach the Marketplace the
two images are not left saying the opposite.
jaymar921
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces HirayaCoder v0.8.0, focusing on improving the agent's ability to avoid redundant actions, enhancing transparency during runs, and fixing several issues with intent detection and session handling. The most important changes include the introduction of a working set that tracks the agent's knowledge, smarter handling of repeated read-only actions, a live step trace panel, and improvements to intent routing and user feedback.
Agent knowledge and action handling:
The agent now maintains a
workingSet(viaworkingSet.js), recording all files and commands it has interacted with. This record is rendered back to the model each turn, reducing unnecessary repeated actions and improving the agent's awareness of its current state. [1] [2] [3] [4] [5] [6] [7] [8]In the ReAct loop, repeated read-only ("reconnaissance") actions no longer immediately end the session. The agent is given the previous result and an explicit instruction on what to do next; only further repetition results in session termination. This prevents premature session endings due to harmless repetitions.
User experience and transparency:
Intent routing and bug fixes:
Documentation and messaging:
These changes collectively make the agent more reliable, transparent, and user-friendly, especially when running on smaller models.