Skip to content

Offload prefetch to mount process for warm auth#2002

Draft
tyrielv wants to merge 1 commit into
microsoft:masterfrom
tyrielv:tyrielv/prefetch-offload-to-mount
Draft

Offload prefetch to mount process for warm auth#2002
tyrielv wants to merge 1 commit into
microsoft:masterfrom
tyrielv:tyrielv/prefetch-offload-to-mount

Conversation

@tyrielv

@tyrielv tyrielv commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Summary

When a GVFS mount is running, gvfs prefetch --commits now sends a PrefetchCommits request to the mount process via named pipe IPC. The mount process executes the prefetch using its already-warm authentication, avoiding the slow cold-auth path (anonymous HTTP probe + git credential helper invocation).

If the mount is not running, not ready, or is an older version that does not recognize the PrefetchCommits message, the verb falls back to the existing direct-auth path transparently.

Motivation

Authentication for prefetch is always slow — the anonymous probe + credential helper invocation adds multiple seconds. The mount process already has warm auth from startup. By offloading to the mount, we skip this cold-auth cost entirely for the common case where the repo is mounted.

Changes

  • NamedPipeMessages.cs: Add PrefetchCommits request/response message types
  • PrefetchStep.cs: Add injectable post-fetch callback parameter to avoid re-entrant named pipe IPC when running inside the mount process
  • InProcessMount.cs: Add HandlePrefetchCommitsRequest handler that runs PrefetchStep synchronously on the connection thread using the mount's warm GVFSGitObjects
  • PrefetchVerb.cs: Try mount offload first; fall back to direct auth only when mount is unavailable (connect failure, MountNotReady, or UnknownRequest from old mount). Mount-side failures are surfaced directly — no fallback on real errors.
  • PrefetchCommitsOffloadTests.cs: Functional tests covering mounted prefetch, idempotent re-prefetch, unmounted fallback to direct auth, and remount scenarios

Design Decisions

  1. Single request/response — no streaming progress in this PR. The spinner covers the wait. Auth speedup is the primary win.
  2. Synchronous on connection thread — connection threads are per-client, won't block other mount IPC. The file-based lock serializes with the mount's own scheduled prefetch.
  3. Post-fetch callback injection — avoids PrefetchStep.SchedulePostFetchJob() opening a named pipe back to the same mount (re-entrant IPC). The mount handler directly enqueues PostFetchStep on its maintenance scheduler.
  4. Fallback only on unavailable — mount-side failure means a real network/auth/pack error; falling back would duplicate work and mask the error.

Testing

  • All 818 unit tests pass
  • Functional tests cover mounted, unmounted, idempotent, and remount scenarios

@tyrielv tyrielv force-pushed the tyrielv/prefetch-offload-to-mount branch 2 times, most recently from a5fb655 to 256710f Compare June 4, 2026 20:16
@tyrielv tyrielv marked this pull request as ready for review June 4, 2026 21:01
@tyrielv tyrielv changed the title Offload prefetch --commits to mount process for warm auth Offload all prefetch operations to mount process for warm auth Jun 4, 2026
@tyrielv tyrielv marked this pull request as draft June 4, 2026 21:06
When a GVFS mount is running, all prefetch operations now offload to the
mount process via named pipe IPC, using its already-warm authentication
to skip the slow cold-auth path (anonymous HTTP probe + git credential
helper invocation).

Commits prefetch (--commits):
  PrefetchCommits IPC message tells the mount to run PrefetchStep with
  its warm GitObjectsHttpRequestor. A post-fetch callback is injected
  to avoid re-entrant named pipe IPC when SchedulePostFetchJob would
  otherwise call back into the same mount.

Blob prefetch (--files/--folders):
  PrefetchBlobs IPC message carries file/folder lists, HEAD commit ID,
  and hydrate flag. The mount creates a fresh GitObjectsHttpRequestor
  with warm auth, runs BlobPrefetcher with capped thread counts
  (ProcessorCount/2), validates inputs, and properly disposes HTTP
  resources. LastBlobPrefetch.dat is passed through for noop state.

Hydration (--hydrate):
  Two-phase approach: mount downloads blobs (no hydrate), then the verb
  process hydrates files locally using Parallel.ForEach with
  ProcessorCount/2 parallelism. This avoids the mount writing to
  ProjFS-virtualized files (self-callback risk) while ensuring all
  blobs are cached before hydration starts, minimizing the ProjFS
  expansion race window.

Fallback:
  If the mount is not running, not ready, or is an older version that
  does not recognize the new IPC messages, the verb falls back to the
  existing direct-auth path transparently. Mount-side failures are
  surfaced directly (no fallback on real errors).

Benchmarks on os.2020 (144 files in tools/nmakejs+Razzle+signing):
  Mounted offload:         158s (warm auth)
  Unmounted direct:        171s (cold auth)
  Mounted offload+hydrate: 153s (two-phase)
  Auth savings: ~13s per prefetch call

Assisted-by: Claude Opus 4.6
Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
@tyrielv tyrielv force-pushed the tyrielv/prefetch-offload-to-mount branch from 4cccafb to df48af7 Compare June 9, 2026 20:25
@tyrielv tyrielv changed the title Offload all prefetch operations to mount process for warm auth Offload prefetch to mount process for warm auth Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant