[ExecuTorch][WebGPU] Fix per-op GPU-timestamp inflation on tile-based mobile GPUs#21033
[ExecuTorch][WebGPU] Fix per-op GPU-timestamp inflation on tile-based mobile GPUs#21033JCNTH wants to merge 2 commits into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21033
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 14 PendingAs of commit 000bc47 with merge base 44539fd ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
SS-JIA
left a comment
There was a problem hiding this comment.
Review automatically exported from Phabricator review in Meta.
… mobile GPUs Pull Request resolved: #21033 The WebGPU per-op profiler (`WebGPUQueryPool::extract_results`) computed each pass's GPU time as `end_of_pass - beginning_of_pass`. On tile-based mobile GPUs (e.g. Arm Mali via Dawn -> Vulkan) the beginning-of-pass timestamps resolve at ~command-buffer start rather than per pass, so each `(end - begin)` measures time-since-start (cumulative) and the summed per-op time inflates by roughly the pass count (~100x on a Llama-1B decode). Desktop backends (Metal) capture beginning-of-pass correctly and are unaffected, which is why it only surfaces on-device. Fix: compute each pass's duration from consecutive end timestamps by clamping each pass's base up to the previous pass's end (`base = max(begin, prev_end)`). On a well-behaved backend `begin >= prev_end`, so `base == begin` and the per-op numbers are byte-unchanged; on a tile-based GPU the clamp recovers the real per-pass time. The summed duration can no longer exceed the wall span by construction. Timestamp fields: `start_time_ns` and `end_time_ns` preserve the raw hardware begin/end ticks; only `execution_duration_ns` carries the consecutive-end correction. Keeping start/end raw makes each record self-consistent (`start <= end`, including the non-monotone-end regime, where the old synthesized `start = base` could exceed `end`), so consumers that align on the raw start/end are unaffected. Limitation: the first dispatch has no earlier end to clamp against, so on a tile GPU its reported duration can still absorb command-buffer work that ran before that pass. Key changes: - `WebGPUQueryPool::fill_shader_durations` (new free function) — sort by dispatch index, then per pass `base = max(begin, prev_end)`, `duration = end > base ? end - base : 0`, tracking `prev_end` as a running max of end timestamps; `start_time_ns`/`end_time_ns` are stored as the raw begin/end ticks. - `WebGPUQueryPool::extract_results` — calls `fill_shader_durations` after mapping the readback (extraction makes the math unit-testable without a GPU). - `#include <algorithm>` for `std::sort` / `std::max`. - `test_webgpu_native.cpp` — device-free unit test `QueryPoolDeltaMath` covering the four regimes, the raw start/end fields, and per-call state reset. Entirely within `#ifdef WGPU_BACKEND_ENABLE_PROFILING`; no change to the non-profiling (production) build. ghstack-source-id: 404723413 @exported-using-ghexport Differential Revision: [D112643054](https://our.internmc.facebook.com/intern/diff/D112643054/)
Stack from ghstack (oldest at bottom):
The WebGPU per-op profiler (
WebGPUQueryPool::extract_results) computed each pass's GPU time asend_of_pass - beginning_of_pass. On tile-based mobile GPUs (e.g. Arm Mali via Dawn -> Vulkan) the beginning-of-pass timestamps resolve at ~command-buffer start rather than per pass, so each(end - begin)measures time-since-start (cumulative) and the summed per-op time inflates by roughly the pass count (~100x on a Llama-1B decode). Desktop backends (Metal) capture beginning-of-pass correctly and are unaffected, which is why it only surfaces on-device.Fix: compute each pass's duration from consecutive end timestamps by clamping each pass's base up to the previous pass's end (
base = max(begin, prev_end)). On a well-behaved backendbegin >= prev_end, sobase == beginand the per-op numbers are byte-unchanged; on a tile-based GPU the clamp recovers the real per-pass time. The summed duration can no longer exceed the wall span by construction.Timestamp fields:
start_time_nsandend_time_nspreserve the raw hardware begin/end ticks; onlyexecution_duration_nscarries the consecutive-end correction. Keeping start/end raw makes each record self-consistent (start <= end, including the non-monotone-end regime, where the old synthesizedstart = basecould exceedend), so consumers that align on the raw start/end are unaffected. Limitation: the first dispatch has no earlier end to clamp against, so on a tile GPU its reported duration can still absorb command-buffer work that ran before that pass.Key changes:
WebGPUQueryPool::fill_shader_durations(new free function) — sort by dispatch index, then per passbase = max(begin, prev_end),duration = end > base ? end - base : 0, trackingprev_endas a running max of end timestamps;start_time_ns/end_time_nsare stored as the raw begin/end ticks.WebGPUQueryPool::extract_results— callsfill_shader_durationsafter mapping the readback (extraction makes the math unit-testable without a GPU).#include <algorithm>forstd::sort/std::max.test_webgpu_native.cpp— device-free unit testQueryPoolDeltaMathcovering the four regimes, the raw start/end fields, and per-call state reset.Entirely within
#ifdef WGPU_BACKEND_ENABLE_PROFILING; no change to the non-profiling (production) build.@exported-using-ghexport
Differential Revision: D112643054
Differential Revision: D112643054