Skip to content

fix(tests): pair same-named function occurrences by line instead of collapsing them - #1533

Merged
squid-protocol merged 1 commit into
mainfrom
fix/tree-sitter-audit-duplicate-name-pairing-1526
Aug 14, 2026
Merged

fix(tests): pair same-named function occurrences by line instead of collapsing them#1533
squid-protocol merged 1 commit into
mainfrom
fix/tree-sitter-audit-duplicate-name-pairing-1526

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Summary

  • tree_sitter_accuracy_audit.py's per-file comparison keyed same-named function occurrences by
    name alone in a plain dict. A file with multiple same-named functions (property getter/setter
    pairs, __init__/__call__ across different classes, repeated methods) silently kept only the
    last tree-sitter occurrence and the first GitGalaxy occurrence -- comparing unrelated
    functions. This wasn't just an args_exact_match problem: real_functions/found_functions
    themselves were undercounted the same way whenever names repeated within a file.
  • Persists function_data.start_line (gitgalaxy/recorders/record_keeper.py -- detector.py
    already computed this internally per function, it just never made it into the DB), and uses it
    in the audit tool to pair same-named real/found occurrences by an order-preserving
    minimum-line-distance alignment (_align_occurrences_by_line) instead of collapsing them onto
    one dict slot.
  • Also excludes detector.py's own synthetic Anonymous_Block/__global_context__ placeholder
    names (used for top-level control-flow blocks in script-style files with no enclosing function)
    from the comparison -- no real language grammar can produce either literal name, so counting
    them as "extra" isn't measuring precision. Left uncorrected this manufactured a wildly misleading
    18.0%/10.7% func precision reading for matlab/shell once same-name collapsing stopped hiding it.
  • Regenerated all 31 language baselines and language_standards.py's summary table to match the
    corrected methodology (this file is shared across every NODE_MAPS language, so the fix's effect
    isn't python-scoped even though tree_sitter_accuracy_audit.py: duplicate function names within a file collapse in the per-file comparison #1526 was found via python).
  • Documented two now-visible categories of "GitGalaxy beats tree-sitter" (Cython cdef class
    scope-loss, MATLAB/shell synthetic-block category mismatch) in
    docs/why_gitgalaxy_beats_ast_here.md, and added a standing CLAUDE.md instruction to log any
    future case there.

Two anomalies filed separately, not fixed here

Regenerating the baselines surfaced two patterns that look like real detector.py extraction
bugs rather than audit-tool artifacts, but are out of this PR's scope (methodology fix, not an
engine-correctness hunt):

Test plan

  • python tests/tools/audit_check.py -- ruff/mypy/dead-key/ast-accuracy all clean
  • python tests/tools/tree_sitter_accuracy_audit.py --all --ci -- all 31 languages pass
  • python tests/tools/tree_sitter_accuracy_audit.py --summary-table -- no diff (already
    regenerated and committed)
  • python -m pytest tests/core_engine/ tests/tools_recorders/ tests/extraction/ tests/cobol_mainframe/ -q -- all pass (2 unrelated tests/security_auditing/ runs timed
    out locally at 100-115s with no failures observed before timeout; unrelated to this PR's
    files, not investigated further)
  • Verified start_line lands correctly end-to-end with a real galaxyscope --db-only scan
    against a hand-built sample file before wiring it into the audit tool
  • Not a parsing-logic change (detector.py/prism.py/language_standards.py's rules are
    untouched; only the summary table, itself derived from baselines, changed) -- no differential
    scan / golden master regen needed

🤖 Generated with Claude Code

…ollapsing them

tree_sitter_accuracy_audit.py's per-file comparison keyed same-named function
occurrences by name alone in a plain dict, so a file with multiple same-named
functions (property getter/setter pairs, __init__/__call__ across different
classes, repeated methods) silently kept only the last tree-sitter occurrence
and the first GitGalaxy occurrence -- comparing unrelated functions. This also
undercounted real_functions/found_functions themselves whenever names repeated
within a file, not just args comparability.

Persists function_data.start_line (record_keeper.py; detector.py already
computed it internally, just never wrote it to the DB) and uses it to pair
same-named occurrences by an order-preserving minimum-line-distance alignment
(_align_occurrences_by_line) instead of collapsing them.

Also excludes detector.py's own synthetic Anonymous_Block/__global_context__
placeholder names (used for top-level control-flow blocks in script-style
files with no enclosing function) from the comparison -- these can never have
a tree-sitter counterpart by construction, so counting them as "extra" isn't
measuring precision. Left uncorrected this manufactured a wildly misleading
18.0%/10.7% func precision reading for matlab/shell in the summary table.

Regenerated all 31 language baselines and the summary table to reflect the
corrected methodology. Two anomalies surfaced during regeneration didn't fit
this fix's scope and are filed separately: Fortran's start_line looking wrong
by ~300 lines for a couple of subroutines (#1531), and Haskell's start_line
looking shifted by roughly one prior function when short definitions are
stacked (#1532) -- both look like real detector.py extraction bugs, not
audit-tool artifacts, and need their own investigation.

Fixes #1526.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol
squid-protocol merged commit 7894f7f into main Aug 14, 2026
30 checks passed
@squid-protocol
squid-protocol deleted the fix/tree-sitter-audit-duplicate-name-pairing-1526 branch August 14, 2026 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant