Skip to content

fix(shell,perl): args-counting for languages with no formal parameter-list syntax - #1520

Merged
squid-protocol merged 1 commit into
mainfrom
fix/shell-perl-args-1517-1518
Aug 14, 2026
Merged

fix(shell,perl): args-counting for languages with no formal parameter-list syntax#1520
squid-protocol merged 1 commit into
mainfrom
fix/shell-perl-args-1517-1518

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Summary

Bash functions and traditional Perl subs have no formal parameter-list syntax at all, permanently, by grammar -- args arrive via body-level idioms instead ($1/$2/"$@" in bash; my (...) = @_ / shift in perl). This measured 0/3 (shell) and 137/937 (perl, 14.6%) on args_exact_match, reading as "GitGalaxy's args regex is badly broken." It wasn't -- both the ground-truth measurement and the engine had real, fixable gaps.

Ground truth (tests/tools/tree_sitter_accuracy_audit.py): _get_param_count only ever checked the declaration for a formal parameter field, found nothing (correctly -- there is none), and reported real=0 regardless of true arity. New _count_shell_real_positional_max / _count_perl_real_args walk the same body idiom GitGalaxy reads, using tree-sitter's own real parse.

Engine (detector.py + language_standards.py):

  • shell: args-counting only ever took the first $N/"$@" match in a function, silently dropping every reference after it. New _args_findall_max_groups routing scans the whole block and takes the max positional index referenced.
  • perl: only counted the first my (...) = @_ or shift statement, missing the extremely common "shift the invocant, then unpack the rest" multi-statement pattern (e.g. bugzilla/Bug.pm::ValidateDependencies, 3 sequential shifts). New _args_findall_sum_groups routing sums every matching statement across the block. Tightened the bare shift alternative to exclude shift @other/shift(@other) (shifting a DIFFERENT array, not @_). Added the bare my @array = @_ variadic catch-all idiom, previously unrecognized entirely.

args_exact_match: shell 0/3 → 3/3 (100%), perl 137/937 → 769/937 (82.1%).

Why this isn't "GitGalaxy loses to tree-sitter"

New doc: docs/why_gitgalaxy_beats_ast_here.md, linked from README.md's "One Graph, Not Five Separate Tools" section. For this one signal, in this one circumstance (no formal signature exists), a declaration-only AST read has nothing to count and can only ever report 0 -- correct, but useless for coupling/complexity scoring. GitGalaxy's body-aware read is the more informative measurement here, not a lower-precision tradeoff. Scoped narrowly -- doesn't apply to languages with real formal signatures, where tree-sitter's ground truth is still the fair, correct comparison (see the six prior fixes cited in the same doc).

Two related findings, filed separately, not fixed here

Verification

tests/extraction/languages/test_shell.py, test_shell_strict.py, test_perl.py, test_perl_strict.py
  -- 260/260 passed (new tests added: shift-vs-shift-other-array exclusion, bare array
  catch-all + ReDoS, multi-statement findall-sum arity via StructuralExtractor, shell
  findall-max via StructuralExtractor)
python -m pytest tests/ -- 6901 passed, 2 skipped, 11 xfailed, 3 xpassed (full suite, clean)
tree_sitter_accuracy_audit --lang shell/perl --regenerate  -- baselines blessed
tree_sitter_accuracy_audit --all --ci  -- 31 languages, zero cross-language regressions
  (both new routing flags are opt-in per-language, gated behind new rules-dict keys)
tests/tools/crucible_check.py --mode both  -- PASS/PASS (golden masters re-blessed; drift is
  real+expected -- perl/shell Function Analysis numbers changed, plus the usual small global
  PageRank/spatial-coordinate ripple across the whole corpus)
tests/tools/audit_check.py --ci  -- ruff/mypy/dead-key/ast-accuracy all clean

Fixes #1518, #1519.

🤖 Generated with Claude Code

…-list syntax

Bash functions and traditional Perl subs have no formal parameter-list
syntax at all, permanently, by grammar -- args arrive via body-level
idioms instead ($1/$2/"$@" in bash; my(...)=@_ / shift in perl). This
was scored 0/3 (shell) and 137/937 (perl, 14.6%) on args_exact_match,
reading as "GitGalaxy's args regex is broken." It wasn't -- both the
ground-truth measurement and the engine had real, fixable gaps:

Ground truth (tests/tools/tree_sitter_accuracy_audit.py): the original
_get_param_count only checked the declaration for a formal parameter
field, found nothing (correctly -- there is none), and reported real=0
regardless of true arity. New _count_shell_real_positional_max /
_count_perl_real_args walk the same body idiom GitGalaxy reads, using
tree-sitter's own real parse.

Engine (detector.py + language_standards.py):
- shell: args-counting only ever took the first $N/"$@" match in a
  function, silently dropping every reference after it. New
  _args_findall_max_groups routing scans the whole block and takes the
  max positional index referenced.
- perl: only counted the first my(...)=@_ or shift statement, missing
  the extremely common "shift the invocant, then unpack the rest"
  multi-statement pattern. New _args_findall_sum_groups routing sums
  every matching statement across the block. Tightened the bare
  "shift" alternative to exclude shift @other/shift(@other) (shifting
  a DIFFERENT array, not @_). Added the bare "my @array = @_" variadic
  catch-all idiom, previously unrecognized entirely.

args_exact_match: shell 0/3 -> 3/3 (100%), perl 137/937 -> 769/937
(82.1%).

Also documents why this isn't "GitGalaxy loses to tree-sitter" --
docs/why_gitgalaxy_beats_ast_here.md, linked from README.md. For this
one signal, in this one circumstance (no formal signature exists), a
declaration-only AST read has nothing to count and can only report 0;
GitGalaxy's body-aware read is the more informative measurement.

Two related findings filed as separate follow-ups, not fixed here:
#1517 (perl func_start block-slicing swallows 432 extra lines for one
real corpus function, pre-existing and unrelated to args-counting) and
a documented, not-fixed nested-anonymous-sub overcounting edge case
(see #1519).

Fixes #1518, #1519.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@squid-protocol
squid-protocol merged commit cb59c23 into main Aug 14, 2026
27 checks passed
@squid-protocol
squid-protocol deleted the fix/shell-perl-args-1517-1518 branch August 14, 2026 00:37
@github-actions

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

shell args: bash has no formal parameter list -- both the ground-truth measurement and GitGalaxy's own counting were incomplete

1 participant