fix(tests): pair same-named function occurrences by line instead of collapsing them - #1533
Merged
squid-protocol merged 1 commit intoAug 14, 2026
Conversation
…ollapsing them tree_sitter_accuracy_audit.py's per-file comparison keyed same-named function occurrences by name alone in a plain dict, so a file with multiple same-named functions (property getter/setter pairs, __init__/__call__ across different classes, repeated methods) silently kept only the last tree-sitter occurrence and the first GitGalaxy occurrence -- comparing unrelated functions. This also undercounted real_functions/found_functions themselves whenever names repeated within a file, not just args comparability. Persists function_data.start_line (record_keeper.py; detector.py already computed it internally, just never wrote it to the DB) and uses it to pair same-named occurrences by an order-preserving minimum-line-distance alignment (_align_occurrences_by_line) instead of collapsing them. Also excludes detector.py's own synthetic Anonymous_Block/__global_context__ placeholder names (used for top-level control-flow blocks in script-style files with no enclosing function) from the comparison -- these can never have a tree-sitter counterpart by construction, so counting them as "extra" isn't measuring precision. Left uncorrected this manufactured a wildly misleading 18.0%/10.7% func precision reading for matlab/shell in the summary table. Regenerated all 31 language baselines and the summary table to reflect the corrected methodology. Two anomalies surfaced during regeneration didn't fit this fix's scope and are filed separately: Fortran's start_line looking wrong by ~300 lines for a couple of subroutines (#1531), and Haskell's start_line looking shifted by roughly one prior function when short definitions are stacked (#1532) -- both look like real detector.py extraction bugs, not audit-tool artifacts, and need their own investigation. Fixes #1526. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
squid-protocol
deleted the
fix/tree-sitter-audit-duplicate-name-pairing-1526
branch
August 14, 2026 01:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tree_sitter_accuracy_audit.py's per-file comparison keyed same-named function occurrences byname alone in a plain dict. A file with multiple same-named functions (property getter/setter
pairs,
__init__/__call__across different classes, repeated methods) silently kept only thelast tree-sitter occurrence and the first GitGalaxy occurrence -- comparing unrelated
functions. This wasn't just an
args_exact_matchproblem:real_functions/found_functionsthemselves were undercounted the same way whenever names repeated within a file.
function_data.start_line(gitgalaxy/recorders/record_keeper.py--detector.pyalready computed this internally per function, it just never made it into the DB), and uses it
in the audit tool to pair same-named real/found occurrences by an order-preserving
minimum-line-distance alignment (
_align_occurrences_by_line) instead of collapsing them ontoone dict slot.
detector.py's own syntheticAnonymous_Block/__global_context__placeholdernames (used for top-level control-flow blocks in script-style files with no enclosing function)
from the comparison -- no real language grammar can produce either literal name, so counting
them as "extra" isn't measuring precision. Left uncorrected this manufactured a wildly misleading
18.0%/10.7% func precision reading for matlab/shell once same-name collapsing stopped hiding it.
language_standards.py's summary table to match thecorrected methodology (this file is shared across every NODE_MAPS language, so the fix's effect
isn't python-scoped even though tree_sitter_accuracy_audit.py: duplicate function names within a file collapse in the per-file comparison #1526 was found via python).
cdef classscope-loss, MATLAB/shell synthetic-block category mismatch) in
docs/why_gitgalaxy_beats_ast_here.md, and added a standingCLAUDE.mdinstruction to log anyfuture case there.
Two anomalies filed separately, not fixed here
Regenerating the baselines surfaced two patterns that look like real
detector.pyextractionbugs rather than audit-tool artifacts, but are out of this PR's scope (methodology fix, not an
engine-correctness hunt):
start_lineis ~300+ lines wrong for a couple of WRF subroutines(
wrf_message,const_module_initialize), withargsalso wrong for the same functions --reads like
func_start's span anchored somewhere other than the real declaration.start_linelooks consistently shifted by roughly one prior function(~10-13 lines) when several short
name :: Type/name args = bodydefinitions are stackedback-to-back.
Test plan
python tests/tools/audit_check.py-- ruff/mypy/dead-key/ast-accuracy all cleanpython tests/tools/tree_sitter_accuracy_audit.py --all --ci-- all 31 languages passpython tests/tools/tree_sitter_accuracy_audit.py --summary-table-- no diff (alreadyregenerated and committed)
python -m pytest tests/core_engine/ tests/tools_recorders/ tests/extraction/ tests/cobol_mainframe/ -q-- all pass (2 unrelatedtests/security_auditing/runs timedout locally at 100-115s with no failures observed before timeout; unrelated to this PR's
files, not investigated further)
start_linelands correctly end-to-end with a realgalaxyscope --db-onlyscanagainst a hand-built sample file before wiring it into the audit tool
detector.py/prism.py/language_standards.py's rules areuntouched; only the summary table, itself derived from baselines, changed) -- no differential
scan / golden master regen needed
🤖 Generated with Claude Code