gather: deterministic context-pack tool (aft_gather)

[iceteaSA]

gather: deterministic context-pack tool (aft_gather)

What

One new tool — aft_gather — assembles a bounded "context pack" (ranked, deduped, budgeted verbatim code evidence) in a single call. It replaces the multi-turn search → outline → zoom → callgraph read chain an agent otherwise runs to build context around a question or a symbol.

Two modes (mutually exclusive):

• question: "how does X work?" — seeds from handle_semantic_search (same pipeline as aft_search, all lanes/fallbacks)
• symbol + filePath — seeds from the callgraph (impact depth-1 callers + call_tree depth-1 callees)

Seeds expand one hop through the callgraph, dedupe by canonicalized (file, symbol) with seeds winning, and render via render_symbol_within_budget until a hard line budget (default 400, cap 800) is spent. Everything past the cut appears as one-line stubs under ## Beyond budget (zoom to expand) — nothing is silently dropped.

Why

Agents burn serial turns assembling context: search, then outline the hits, then zoom the symbols, then chase callers. Each turn round-trips through the model. A pack returns evidence (verbatim bodies with file:line headers), not conclusions — the agent reasons over it directly, ready to attach to a subagent dispatch.

Measured on a real config repo (3 questions, one call each vs. the manual chain):

• "how does the lane-verdict cache flow end-to-end" → complete 4-file chain (cache shapes, verdict logic, write/read, plugin deny hook) in one pack, used=283/400. Manual baseline: 5-6 tool calls.
• "how does score-tap decide when to write an executor row" → the full decision chain (event filter → dedup → context sources → verdict parse → DB insert) in one pack at budget=200.
• An unrehearsed cross-file question → correct core symbols plus their docstrings, first try.

Independently reproduced on the aft codebase itself: question: "how does bash output compression dispatch pick a compressor" → seeds=15, used=226/400, one pack assembling the full dispatch chain (compress → gate → compress_with_registry_exit_code 20-compressor array → Compressor trait → install path → subc mirror) that otherwise takes a 4–5-call search→zoom chain.

Honest degradation

The pack never lies about its own quality:

• While the semantic index is building, long NL queries degrade to lexical-only file-level hits. The pack renders them as visible file:line (no containing symbol) stubs and flags the header with degraded=semantic-index-building (partial results — retry when index ready) — detected via the response's semantic_status field, cleared as soon as one real seed resolves. No blocking or retry inside the tool.
• Grep-fallback hits ({file, line_text, line}, no symbol name) resolve to their containing symbol by line containment — definitions, call sites, and comment hits all upgrade to the enclosing symbol. Hits with no containing symbol stay visible as stubs.
• Unresolved external/stdlib callees collapse to one summary line ((N unresolved external calls omitted)) instead of drowning the stub list; unresolved seeds and callers are never suppressed.

Implementation

• crates/aft/src/commands/gather.rs — Rust-side composition: calls handle_semantic_search / impact_result / call_tree_result / render_symbol_within_budget directly (shared &AppContext, no bridge round-trips, no parallel reimplementation of search).
• Wiring: main.rs dispatch arm, subc_translate.rs mapping, TS factory packages/opencode-plugin/src/tools/gather.ts + registration (same tier as aft_callgraph — depends on the callgraph store).
• No new dependencies, no config surface beyond the tool args, no LLM calls, no caching.
• Follows the tri-state honest-reporting convention (protocol.rs Response doc-comment): success:false+code for un-performable calls (e.g. invalid_request on a bad mode combo), success:true with a visible degraded/stub pack for partial results — never a bare empty success.

Tests

22 unit tests in gather.rs, including red-checked regressions (each confirmed to fail against pre-fix code): mid-codepoint truncation panic, duplicate-symbol line-anchored resolution, abs/rel path dedupe, containing-symbol resolution via a real TreeSitterProvider, callee-only stub suppression driven through the production build_pack path, and degradation-flag presence/absence/mixed cases.

Limitations (deliberate scope)

• 1-hop expansion only — multi-hop was deliberately excluded: the budget math and ranking get harder and the packs get noisier.
• Callgraph-dependent by design: neighbor expansion quality follows the callgraph store's freshness, same as aft_callgraph.
• Budget counted in lines, not tokens — matches aft's existing budget idiom across zoom/outline.

---

^{Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.}

---

Summary by cubic

Adds aft_gather, a one-call, deterministic context-pack builder that returns ranked, deduped, verbatim code within a fixed line budget, replacing the multi-step search→outline→zoom→callgraph chain. Exposed as gather in Rust and aft_gather in @opencode (surface: "all").

• New Features

  • Two modes: question (semantic seeds) or symbol + filePath (impact callers + call-tree callees), with 1‑hop expansion and (file, symbol) dedupe (seeds win). Mode XOR validated in Rust and the plugin schema.
  • Hard line budget (default 400, max 800); overflow goes under “Beyond budget” as stubs. External unresolved callees collapse to a single summary line; seeds/callers remain visible.
  • Grep‑fallback hits resolve to containing symbols; hits without one render as visible stubs.
  • Header flags degraded=semantic-index-building (suppressed once any symbol seed resolves) and neighbors=skipped(callgraph-unavailable) when the callgraph store is down.
  • Robustness: repo‑relative paths resolve against project root (incl. filePath in translate); abs/rel path normalization for dedupe; Unicode‑safe query truncation. No new deps. 22 tests. README adds aft_gather. Tool gated in ALL_ONLY_TOOLS.

• Migration

  • Call with { question: "how does X work?" } or { symbol, filePath }; optional budget 1–800 (default 400).
  • Neighbor expansion requires the callgraph store in symbol mode; in question mode, neighbors are skipped with a header notice if unavailable.

^{Written for commit f42c5ec. Summary will update on new commits.}

Greptile Summary

Adds aft_gather, a one-call deterministic context-pack builder that replaces the multi-turn search → outline → zoom → callgraph chain. The implementation is well-structured and carefully handles degradation, deduplication, Unicode truncation, and budget accounting.

• Question mode seeds from handle_semantic_search (same pipeline as aft_search), expands one hop via the callgraph, dedupes by canonicalized (file, symbol), and renders within a hard line budget; grep-fallback hits with no containing symbol are promoted to visible stubs rather than silently dropped.
• Symbol mode seeds from impact callers + call-tree callees for the given (symbol, filePath) pair, requiring the callgraph store to be ready.
• Budget accounting correctly counts section lines plus the per-section separator newline; the used= header is built after rendering to avoid false matches inside query text, and degradation flags are suppressed once any symbol-level seed resolves.

Confidence Score: 5/5

Safe to merge — the new gather command adds a self-contained code path with no changes to existing handlers.

Budget accounting is correct (section lines + per-section separator + header). The callgraph store's own normalize_file_path joins relative paths with project_root so neighbor expansion works in both modes. Deduplication is correct with seeds winning. Degradation flags are suppressed once any seed resolves. The 22 unit tests cover the key regression cases. No correctness defects were found in the changed code.

No files require special attention.

Important Files Changed

Filename	Overview
crates/aft/src/commands/gather.rs	Core implementation of aft_gather. Budget accounting, dedup, degradation flags, Unicode truncation, and path normalization are all correct. 22 unit tests cover edge cases including mid-codepoint truncation, abs/rel path dedup, and callee-only suppression. One dead code loop (orphan-neighbor check) and a misleading doc comment on hop_distance.
packages/opencode-plugin/src/tools/gather.ts	Thin TypeScript adapter with correct XOR mode validation mirroring translate_gather. Schema uses zod with min/max constraints on budget. Error handling correctly surfaces Rust-side error messages.
crates/aft/src/subc_translate.rs	New translate_gather resolves filePath against project_root and validates mode XOR with granular errors. Consistent with adjacent translate functions.
crates/aft/src/main.rs	Minimal change: adds 'gather' dispatch arm alongside 'impact'. No issues.
packages/opencode-plugin/src/index.ts	Adds aft_gather to ALL_ONLY_TOOLS (same tier as aft_callgraph) and spreads gatherTools unconditionally (filtered at surface check). Registration is consistent with the callgraph tool pattern.

Sequence Diagram

sequenceDiagram
    participant Agent
    participant TS as gather.ts
    participant TR as translate_gather
    participant HG as handle_gather
    participant SS as handle_semantic_search
    participant CG as callgraph_store
    participant SR as render_symbol_within_budget

    Agent->>TS: "aft_gather({question|symbol+filePath, budget})"
    TS->>TS: XOR mode validation
    TS->>TR: callToolCall(gather, rawArgs)
    TR->>TR: resolve filePath to absolute
    TR->>HG: "RawRequest{command:gather}"

    alt question mode
        HG->>SS: "handle_semantic_search(query, top_k=15)"
        SS-->>HG: "results[{file, name, score}]"
        HG->>HG: resolve grep-fallback hits via resolve_containing_symbol
        HG->>CG: impact_result + call_tree_result (1-hop)
        CG-->>HG: callers + callees per seed
    else symbol mode
        HG->>CG: "validate_path then impact_result(depth=1)"
        CG-->>HG: callers
        HG->>CG: "call_tree_result(depth=1)"
        CG-->>HG: callees
    end

    HG->>HG: dedup by (file,symbol) seeds win
    loop each candidate
        HG->>SR: render_symbol_within_budget(per_symbol_budget)
        SR-->>HG: content + status
        HG->>HG: budget check to body or stubs
    end

    HG->>HG: "build header with used=N and flags"
    HG-->>TS: "Response{success:true, text:pack}"
    TS-->>Agent: pack text

Loading

_{Reviews (9): Last reviewed commit: "gather: deterministic context-pack tool ..." | Re-trigger Greptile}

[cubic-dev-ai]

4 issues found across 7 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[iceteaSA]

Follow-up on the two maintainability notes from the Greptile summary (they weren't separate review threads, so noting here) — both addressed in cf09b9b9:

• degraded expression flagged as always-false in the non-empty-seeds path — intentional: degraded only applies when zero symbol-level seeds resolve (a building index yields no seeds; a ready index with seeds is never degraded). Logic unchanged; added a comment on the why so it doesn't read as dead code.
• callee-suppression string-match coupling — extracted UNRESOLVED_MARKER ("(symbol not resolved)") and CALLEE_PROVENANCE_PREFIX ("callee-of-") consts, referenced by both the producer (render_symbol_section err arm / collect_callees_for_seed) and the consumer (build_pack suppression guard) so the match can't silently drift. Only test-site literals remain, intentionally — a test pointing at the const couldn't catch const drift.

Matched/rendered text is byte-identical; 23/23 gather tests green.