feat(r): extract roxygen cross-references and S3 dispatch - #2395
feat(r): extract roxygen cross-references and S3 dispatch#2395fernando-duarte wants to merge 2 commits into
Conversation
R was in CODE_EXTENSIONS but had no entry in _DISPATCH, so every .R file was counted as code and then contributed nothing — it is the language Graphify-Labs#1689's no-AST-extractor warning names as its example. R has no named-function syntax: `f <- function(x)`, `f = \(x)` and `(function(x) x) -> f` are all assignments binding an anonymous function_definition, whose only `name` field is the `function` keyword itself. The config-driven walker reads a name off the function node, so this is a bespoke extractor. Calls are resolved corpus-wide rather than per file. Within a package or an analysis directory R has one shared namespace and no import statement binding a name to a file, so `paste0(x)` and a sibling file's `compute_moments(x)` are the same syntax and the file cannot tell them apart. The extractor emits unresolved calls as raw_calls and graphify.r_resolution links a callee only when the corpus defines it exactly once — the god-node guard the Java and ObjC resolvers use. Everything else is dropped, so base R stays out of the graph without hardcoding ~1,300 base names and no dangling edge reaches build_from_json. Sourced files resolve the same way: any R path literal, including one joined from string literals by a helper (`source_once(paper_path("sub", "b.R"))`), links only when it matches exactly one corpus file. The r-lib grammar has no standalone PyPI wheel, so it comes from tree-sitter-language-pack under a new optional [r] extra, hard-failing like tree-sitter-sql rather than falling back. Two tests in test_extract.py used .r as their example of a code extension with no extractor; they move to .ets, which is now one of the two that remain. Measured on a 541-file R corpus: 2,171 nodes and 7,434 edges in 2.2s, zero dangling edges, and no base-R hub in the top-degree nodes.
Roxygen lives in `#'` comments, so the AST cannot see it, and the tags carry relationships nothing else in R does. `@seealso` and `@inheritParams` name functions the documented one never calls — exactly what a call graph structurally cannot find. `@template` names a file under man-roxygen/ that nothing sources, so those files were corpus orphans. `@export` is the only thing separating the public API from an internal helper. S3 dispatch leaves no syntax behind: `print.hetid_moments` is a method only because `print` is a generic and `hetid_moments` is a class, and neither fact is in the file defining the method. Every dot is emitted as a candidate split and r_resolution keeps only what the corpus evidences — a `UseMethod` generic, or a site assigning that class — so `is.null` and `compute.stuff` fabricate nothing. A class is often re-attached after a transformation, leaving several sites that assign it; R's `new_<class>` constructor convention breaks that tie. Non-call records carry `ref_name`, not `callee`. The shared cross-file pass turns any raw_call with a `callee` into a `calls` edge by name, so `@seealso` shipped as a phantom call AND blocked the real `references` edge as a duplicate — the same trap Ruby's mixin markers hit in Graphify-Labs#1668. On a 550-file R corpus this moved 18 edges from `calls` to `references`, where they belong. Measured on that corpus: 92 doc edges (74 @template, 18 @seealso/@inheritParams), 2 S3 method edges, 35 functions marked exported, zero dangling edges.
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify review — findings
This PR adds R language support to the graphify extractor pipeline. It introduces a new bespoke extract_r extractor (registered for .r/.R extensions and Rscript shebangs) along with roxygen block parsing and S3 dispatch handling, plus a corpus-wide resolve_r_calls resolver registered in the resolver framework. It also updates the dispatch tables, extractor registry, the [r] optional extra mapping, and revises the no-AST-extractor warning comments that previously used R as their example, along with changelog entries and accompanying tests.
No blocking issues surfaced. 1 lower-confidence candidate did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 1659 functions depend on the 721 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
extract()— 370 callers, 29 callees - worse:
_get_extractor()— 26 callers, 6 callees - new:
extract_r()— 14 callers, 4 callees - new:
walk_calls()— 0 callers, 11 callees - new:
collect_definitions()— 0 callers, 8 callees
Verification — 1659 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 1531 function(s) in the blast radius were not formally verified this run
· 3 grounded finding(s) anchored inline below; 2 more finding(s) on lines outside this diff (see the check run).
| return joined if joined.lower().endswith((".r",)) else None | ||
|
|
||
|
|
||
| def extract_r(path: Path) -> dict: |
There was a problem hiding this comment.
extract_r()
14 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return rhs, lhs | ||
| return None | ||
|
|
||
| def collect_definitions(node, scope_nid: str) -> None: |
There was a problem hiding this comment.
collect_definitions()
fans out to 8 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| add_node(imp_nid, name, line) | ||
| add_edge(scope_nid, imp_nid, "imports", line, context="import") | ||
|
|
||
| def walk_calls(node, caller_nid: str) -> None: |
There was a problem hiding this comment.
walk_calls()
fans out to 11 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
1 participant
Roxygen
Roxygen lives in
#'comments, so the AST cannot see it — tree-sitter reports a block as a run ofcommentnodes with no link to the binding below. The tags carry relationships nothing else in R does:@seealso/@inheritParamsname functions the documented one never calls, which is exactly what a call graph structurally cannot find.@templatenames a file underman-roxygen/that nothing sources — roxygen splices it at document time — so those files were corpus orphans.@exportis the only thing separating the public API from an internal helper; it is recorded on the node, as is@family.Blocks are matched to code by line, because roxygen requires the block to sit directly above its object. Parsing is line-oriented for the same reason: it is how roxygen itself reads them.
S3
S3 dispatch leaves no syntax behind.
print.hetid_momentsis a method only becauseprintis a generic andhetid_momentsis a class, and neither fact is in the file that defines the method. So the extractor emits every dot position as a candidate generic/class split andr_resolutionkeeps only what the corpus evidences:UseMethodgeneric of that name — the definitive marker, preferred; orstructure(..., class = "x"),class(x) <- "x").Neither present ⇒ nothing emitted, so
is.null,as.matrixandcompute.stufffabricate no method edges. A class is often re-attached after a transformation, leaving several sites that assign it; R'snew_<class>constructor convention breaks that tie.The bug this surfaced
The shared cross-file pass turns any raw_call carrying a
calleeinto acallsedge by name. A@seealsorecord therefore shipped as a phantomcallsedge and then blocked the realreferencesedge as a duplicate — the same trap Ruby's mixin markers hit in #1668, which that code comments on directly.Fixed by naming the field
ref_nameon non-call records, so they are invisible to that pass (if not callee: continue) and onlyr_resolutionreads them. No shared code changed. On a 550-file R corpus this moved 18 edges out ofcallsand intoreferences, where they belong. A test pins it.Measured
Same 550-file R corpus (an R package plus a 400-file analysis pipeline):
@templateedges@seealso/@inheritParamsedgesexportedWorth stating plainly: on this corpus every
@seealsotarget is a function the documenting function already calls, so those 18 are reclassifications rather than new pairs. The value shows up on codebases where@seealsopoints somewhere the call graph does not reach.Tests
Nine added to
tests/test_r.py(23 total): every binding form still passes, plus@exportmarking,@family, seealso/template emission, cross-file@seealsoresolution,@template→man-roxygen/resolution, S3 via class constructor, S3 preferring aUseMethodgeneric, an ordinary dotted name producing nothing, and the no-callee-on-non-call-records invariant.Full suite: same 15 failures as base commit
00efd6e(11test_skillgen, 4test_ollama_retry_cap— pre-existing, unrelated), 3,915 passed.