Update: improve scope stats reporting

[zmnobug]

Summary

• Add the dep_pool resource channel to scope_stats collection and reporting.
• Improve the scope_stats HTML for top peaks, resource pressure charts, hover details, and expanded chart views.
• Update scope_stats documentation to explain the displayed fields and report interpretation.

Fixes #991

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR extends the scope-stats collection system to track dependency-pool occupancy alongside existing task-window, heap, and tensormap metrics. Changes span data contracts (adding dep_pool fields to ScopeStatsRecord and ScopeStatsDataHeader), scheduler snapshot publication infrastructure, runtime capture during scope boundaries, collector ABI/storage updates, JSONL serialization (schema version 5), and an extensive HTML report generator refactor with interactive charts and risk-metric visualization.

Changes

Scope-stats dependency-pool resource integration

Layer / File(s)	Summary
Data contracts and platform APIs src/common/platform/include/common/scope_stats.h, src/common/platform/include/aicpu/scope_stats_collector_aicpu.h, src/common/platform/include/host/scope_stats_collector.h	ScopeStatsRecord and ScopeStatsDataHeader structs gain dep_pool_start, dep_pool_end, and dep_pool_cap[] fields; exported C functions scope_stats_begin, scope_stats_end, and scope_stats_set_ring_capacity are updated with new dep_pool parameters.
Scheduler snapshot publication and reading src/a2a3/runtime/.../scheduler/pto_scheduler.h, src/a2a3/runtime/.../shared/pto_runtime2_init.cpp, src/a5/runtime/.../scheduler/pto_scheduler.h, src/a5/runtime/.../shared/pto_runtime2_init.cpp	RingSchedState adds atomic dep_pool_snapshot_tail/top fields with publish_dep_pool_snapshot() and read_dep_pool_snapshot() helpers. Snapshots are published after reclaiming entries in drain_wiring_queue and after wiring tasks, and initialized to 1 during ring setup.
Runtime scope boundary capture src/a2a3/runtime/.../aicpu/aicpu_executor.cpp, src/a2a3/runtime/.../pto_orchestrator.cpp, src/a5/runtime/.../aicpu/aicpu_executor.cpp, src/a5/runtime/.../pto_orchestrator.cpp	Executor passes dep_pool_capacity to scope_stats_set_ring_capacity during initialization. Orchestrator begin/end scope methods read dep_pool snapshot from scheduler and pass dep_pool_start/end to scope_stats_begin/end alongside task/heap and tensormap metrics.
Collector implementation and JSONL schema src/common/platform/shared/aicpu/scope_stats_collector_aicpu.cpp, src/common/platform/shared/host/scope_stats_collector.cpp	Collector functions accept and store dep_pool_start/end in ScopeStatsRecord. NDJSON output upgraded to schema version 5 with dep_pool_max metadata array and per-record dep_pool_start/end fields; serialization buffers increased to accommodate larger payloads.
Interactive HTML report generation simpler_setup/tools/scope_stats_plot.py	Resource data model refactored with Resource and Metric NamedTuples; dep_pool resource added with risk-metric selection. Report now includes header summary (site/status counts), Formula Guide (metric descriptions and formulas), Top Peaks table (capacity/use visualization), and interactive multi-series SVG charts with grid/axes, hover tooltips, peak markers, and keyboard/click-driven modal enlargement (CSS and inline JavaScript).
Documentation and test expectations docs/dfx/scope-stats.md, tests/st/a2a3/.../test_scope_stats.py, simpler_setup/scene_test.py, src/common/task_interface/call_config.h	Documentation updated to schema version 5 with dep_pool fields; quick-start path now scope_stats/scope_stats.jsonl; "Reading the report" section expanded with panel/chart interaction details. Test expectations updated to require version 5 metadata and dep_pool_start/end fields. Output path references updated across all config/documentation.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related issues

• hw-native-sys/simpler#991: Both sets of changes implement the same feature-level and code-level work: adding a dep_pool resource channel and associated parsing/metrics/HTML-reporting (scope_stats HTML generator and related scope_stats APIs/records), so they are directly related.

Possibly related PRs

• hw-native-sys/simpler#858: Main PR's dep-pool–aware scope profiling extends the existing scope_stats collector ABI/output schema (e.g., scope_stats_set_ring_capacity, scope_stats_begin/end, and JSONL fields) in the same subsystem introduced by PR #858, so the changes build directly on that implementation.

Poem

A pool of deps waits to be seen,
Now sampled clean and tracked serene. 🐇
From snapshots true through rings it flows,
Charts bloom where occupancy shows. 📊

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Update: improve scope stats reporting' directly summarizes the main objective of the pull request—enhancing scope stats reporting by adding dep_pool tracking and refreshing the HTML report UI/documentation.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	The pull request description accurately summarizes the changeset, mentioning dep_pool resource addition, scope_stats HTML improvements, and documentation updates.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request extends the scope_stats diagnostic feature to track and visualize scheduler dependency-list pool (dep_pool) usage, updating the schema to version 5. Key changes include capturing dep-pool snapshots in the runtime and scheduler, passing these metrics through the AICPU collector, and enhancing the HTML report generator with new chart stacks, a formula guide, and a top peaks table. The review feedback highlights several improvement opportunities, including resolving a potential ValueError crash in the plotting tool when finding the minimum of an empty sequence, defensively using .get() to prevent KeyError on malformed records, adding explicit padding in the ScopeStatsDataHeader shared memory structure to avoid compiler-dependent misalignment, and replacing angle brackets in Markdown placeholders to prevent shell syntax errors.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

simpler_setup/tools/scope_stats_plot.py (1)

124-138: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Use LIFO pairing for same-site nested scopes.

At Line 135, FIFO matching (pop(0)) mispairs begin/end when the same site is nested (e.g., recursive/re-entrant scope usage), which corrupts derived metrics and peak attribution. Use stack semantics (pop()).

Suggested fix

-            queue = pending[ring].get(site)
-            if queue:
-                pairs[ring].append((queue.pop(0), rec))
+            stack = pending[ring].get(site)
+            if stack:
+                pairs[ring].append((stack.pop(), rec))

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@simpler_setup/tools/scope_stats_plot.py` around lines 124 - 138, The pairing
logic in _pair_by_ring uses FIFO queue semantics (pending[ring][site].pop(0))
which mispairs nested/re-entrant same-site scopes; change it to LIFO stack
semantics by using pending[ring][site].pop() when matching an "end" record
(update the code around the queue variable and the pop call) so the most recent
"begin" pairs with the current "end".

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@simpler_setup/tools/scope_stats_plot.py`:
- Around line 124-138: The pairing logic in _pair_by_ring uses FIFO queue
semantics (pending[ring][site].pop(0)) which mispairs nested/re-entrant
same-site scopes; change it to LIFO stack semantics by using
pending[ring][site].pop() when matching an "end" record (update the code around
the queue variable and the pop call) so the most recent "begin" pairs with the
current "end".

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7a6cfc96-6ea0-482e-9861-f5e028037419

📥 Commits

Reviewing files that changed from the base of the PR and between f257850 and ac3e67a.

📒 Files selected for processing (18)

• docs/dfx/scope-stats.md
• simpler_setup/scene_test.py
• simpler_setup/tools/scope_stats_plot.py
• src/a2a3/runtime/tensormap_and_ringbuffer/aicpu/aicpu_executor.cpp
• src/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_orchestrator.cpp
• src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/pto_scheduler.h
• src/a2a3/runtime/tensormap_and_ringbuffer/runtime/shared/pto_runtime2_init.cpp
• src/a5/runtime/tensormap_and_ringbuffer/aicpu/aicpu_executor.cpp
• src/a5/runtime/tensormap_and_ringbuffer/runtime/pto_orchestrator.cpp
• src/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/pto_scheduler.h
• src/a5/runtime/tensormap_and_ringbuffer/runtime/shared/pto_runtime2_init.cpp
• src/common/platform/include/aicpu/scope_stats_collector_aicpu.h
• src/common/platform/include/common/scope_stats.h
• src/common/platform/include/host/scope_stats_collector.h
• src/common/platform/shared/aicpu/scope_stats_collector_aicpu.cpp
• src/common/platform/shared/host/scope_stats_collector.cpp
• src/common/task_interface/call_config.h
• tests/st/a2a3/tensormap_and_ringbuffer/dfx/scope_stats/test_scope_stats.py

[zmnobug]

Addressed the CodeRabbit outside-diff item in 9baeb05: _pair_by_ring now uses LIFO matching for same-site nested scopes, with a unit test covering the nested pairing case.