Phase1: tree sitter dependency extraction for diffgraph#13
Open
avikalpg wants to merge 7 commits into
Open
Conversation
- Add TreeSitterProcessor with multi-language support framework - Implement Python component extraction (classes, functions, methods) - Use tree-sitter-language-pack for AST parsing - Support Python, TypeScript, JavaScript, Go, Rust, Java, Swift (Python fully working) - Extract component relationships from static analysis - Register as 'tree-sitter-dependency-graph' mode - Add basic test demonstrating Python extraction Component extraction includes: - Classes/containers with nested methods - Standalone functions - Import statements - Function call analysis for dependency mapping Successfully tested with Python code showing proper extraction of: - 1 class (MyClass) - 2 methods (__init__, increment) - 2 functions (standalone_function, another_function) Next steps: - Complete TypeScript/JavaScript extraction (partial implementation) - Complete Go, Rust, Java, Swift extraction (partial implementation) - Add comprehensive testing across all supported languages - Enhance dependency detection for cross-file references
- Add tree-sitter-dependency-graph to available modes in README - Document Python support as fully working - Note other languages as in progress - Update CHANGELOG with phase 1 implementation details - Add usage examples for tree-sitter mode
- Document completed work: Python extraction fully functional - Detail in-progress work for other languages - Explain tree-sitter API migration (0.25+) - Provide technical insights and lessons learned - Outline next steps for phases 2-4 - Include usage examples and performance notes
Move documentation files to docs/: - PHASE1_SUMMARY.md -> docs/ - GRAPH_EXPORT_FEATURE.md -> docs/ - TESTING_GUIDE.md -> docs/ Move test files to tests/: - test_tree_sitter_basic.py -> tests/ - test_graph_export.py -> tests/ - test_structured_export.py -> tests/ - test_cli_manual.sh -> tests/ Move example to tests/examples/: - example_usage.py -> tests/examples/ This improves project organization and follows standard conventions.
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (2)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
4 tasks
added 3 commits
July 6, 2026 22:03
Addresses the two blocking gaps from PR-13-REVIEW.md: Gap 1 — Output format: new schema_v2_adapter.py module produces symbols[] + relationships[] with analysis_source='structural' and evidence pointers, conforming to diffgraph-v2.schema.json. Gap 2 — Symbol diff layer: compute_symbol_diff() compares pre-change and post-change AST snapshots to assign change_kind (added / modified / deleted / unchanged) to every symbol. Uses content-slice comparison so symbols that shift in line number without changing body are correctly marked 'unchanged'. Also adds: - _get_pre_change_content() / _get_post_change_content() to TreeSitterProcessor (Gap 7 — explicit pre/post split) - analyze_changes_v2() entry point on TreeSitterProcessor, which calls the adapter and returns a schema v2 dict. No GraphManager, no Mermaid, no network calls. metadata.privacy_tier='local' always. - Guard openai_agents_dependency import with try/except so the module loads without the openai-agents SDK installed. - 21 deterministic unit tests (test_schema_v2_adapter.py), including a zero-network-calls assertion (Gap 6).
…inst diffgraph-v2.schema.json
Five concrete gaps fixed:
1. _get_parser: was calling tslp.get_parser() (incompatible native API); now
uses ts.Parser(tslp.get_language()) so node.type / node.start_point /
node.children all work correctly. This was silently swallowed by except/pass,
causing zero symbols and zero imports on every run.
2. build_symbol_entry: added required schema v2 fields (name, file_id, kind,
parent_id); renamed file→file_id; flattened evidence location (was nested
under location{}, schema expects flat file/line_start/line_end on evidence).
3. build_import_relationship: renamed from/to → source_id/target_id; added id
field with format rel::<src>-><tgt> matching schema pattern requirement.
4. build_schema_v2_output: diff_ref had disallowed from/to fields (schema has
additionalProperties:false); warnings was at top-level (schema has it in
metadata.warnings); files entries were missing id and analysis_source.
5. tests: updated test_schema_v2_adapter assertions for new field names;
updated test_tree_sitter_basic to use schema v2 dict API (not old
DiffAnalysis object).
Result: 45 tests pass (up from 40); all Phase 1 acceptance criteria met:
- analyze_changes() returns schema v2 dict
- privacy_tier == local
- output validates against diffgraph-v2.schema.json
- relationships[] includes import relationships for Python
- metadata.analysis_duration_ms present
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 1: Tree-sitter Dependency Extraction + Schema V2 Output
This PR implements the tree-sitter AST-based dependency extraction processor and wires it to produce schema v2 output — the foundational layer for DiffGraph v2.
What's included
Phase 1 acceptance criteria (all met)
TreeSitterProcessor.analyze_changes()returns schema v2 dict (not GraphManager)privacy_tier == "local"diffgraph/schema/diffgraph-v2.schema.jsonrelationships[]includes import relationships for Python filesmetadata.analysis_duration_msis presentKey fixes (from latest commits)
_get_parser: was using incompatible native API; now usests.Parser(tslp.get_language())build_symbol_entry: added required schema v2 fields (name, file_id, kind, parent_id)build_import_relationship: renamed from/to → source_id/target_id; added id fieldbuild_schema_v2_output: fixed diff_ref schema violations, moved warnings into metadataWhat this enables
Once merged:
wild diffcan emit schema v2 JSON (structural tier) from local static analysis — no API call neededMerge sequence context
This is Phase 1 in the v2 implementation roadmap. After this merges:
→ Phase 2 fixes go on
feature/multiple-processing-modes(PR #12)→ Phase 3 fixes go on
feature/output-graph-data(PR #11)→ Phase 4 is a new branch (terminal formatter)
See
docs/DiffGraph-CLI/design/V2-IMPLEMENTATION-ROADMAP.mdfor full plan.