Skip to content

Phase 9: trackpad volume gesture (shared F4) - #9

Open
tirthfx wants to merge 19 commits into
phase-8-lockscreen-lyricsfrom
phase-9-volume-gesture
Open

Phase 9: trackpad volume gesture (shared F4)#9
tirthfx wants to merge 19 commits into
phase-8-lockscreen-lyricsfrom
phase-9-volume-gesture

Conversation

@tirthfx

@tirthfx tirthfx commented Jul 10, 2026

Copy link
Copy Markdown
Owner

What changed

Phase 9 — the shared trackpad left-edge volume gesture (PRD §3.3 / F4), the hardest phase. A two-finger vertical slide that starts at the trackpad's physical left edge acts as a system-wide volume fader — regardless of cursor position or which surface (notch / lock screen / neither) is visible.

  • MultitouchSupport.swift — runtime bindings to the private framework via dlopen/dlsym (no public header, can't link directly). MTTouch uses the long-standing community-reverse-engineered 96-byte layout (cited in the source comment); offsets through normalized are the stable part and all we read. The loader asserts the stride (verified = 96, normalized @ offset 32) and disables the gesture gracefully if a future OS changes it, rather than striding through touches incorrectly.
  • VolumeController.swift — reads/sets kAudioHardwareServiceDeviceProperty_VirtualMainVolume on the current default output device via Core Audio, so speakers / headphones / AirPlay all work; re-resolves the device each call (it can change).
  • GestureEngine.swift — arms only when both fingers land in the left-edge strip (x < 0.16), which is what separates a deliberate edge slide from ordinary two-finger scrolling (which starts anywhere); a dead-zone before engaging avoids twitch; vertical delta maps to volume with throttled Core Audio writes. The bare C callback (no context) routes through a shared instance.
  • Opt-in menu toggle "Left-edge Volume Gesture"; enabling prompts for Input Monitoring (IOHIDRequestAccess) the first time. Off by default so nothing claims trackpad space or triggers a permission prompt unexpectedly at launch.

Why

Completes the shared gesture layer from the brief — the notch/lock-screen surfaces handle now-playing; this adds the always-on volume fader. The app is unsandboxed (no App Sandbox entitlement), which is what permits the private-framework dlopen and Core Audio calls (PRD §4 already notes this can't ship on the Mac App Store).

Base

Top of the stack: 5 → 6 → 7 → 8 → 9, based on phase-8-lockscreen-lyrics (open PR #8). Diff is Phase 9 only.

Notes for review — needs your eyes + hands on the trackpad

  • Can't be verified headlessly (needs a real trackpad + the Input Monitoring grant, and the multitouch callback only fires on live hardware). Confirmed: builds clean, launches without crashing, MTTouch stride == 96 checked against the C layout, notch/lock screen unaffected.
  • Please test on the Mac: menu → enable Left-edge Volume Gesture → grant Input Monitoring when prompted → slide two fingers up/down starting at the far-left edge of the trackpad → system volume should move smoothly; normal two-finger scrolling elsewhere should NOT change volume.
  • Tuning knobs if it feels off, all constants at the top of GestureEngine: edgeThreshold (how wide the left strip is), travelForFullRange (sensitivity), deadZone. Slide direction is natural (up = louder) — flip the sign in handleFrame if you'd prefer otherwise.
  • If MultitouchSupport can't start (e.g. permission denied), it logs and no-ops — the toggle just won't stick on.

🤖 Generated with Claude Code

System-wide two-finger left-edge slide → system volume (PRD §3.3 / F4), the
hardest phase. Works regardless of cursor position or which surface is visible.

- MultitouchSupport.swift: runtime bindings to the private framework via
  dlopen/dlsym (no public header, can't link directly). MTTouch uses the
  community-reverse-engineered 96-byte layout (offsets through 'normalized' are
  stable); the loader asserts the stride and bails gracefully otherwise.
- VolumeController.swift: reads/sets kAudioHardwareServiceDeviceProperty_
  VirtualMainVolume on the current default output device via Core Audio, so
  speakers / headphones / AirPlay all work.
- GestureEngine.swift: arms only when BOTH fingers land in the left-edge strip
  (x < 0.16), which separates a deliberate edge slide from ordinary two-finger
  scrolling; dead-zone before engaging; vertical delta → volume with throttled
  Core Audio writes. The bare C callback routes through a shared instance.
- Opt-in menu toggle 'Left-edge Volume Gesture'; enabling prompts for Input
  Monitoring (IOHIDRequestAccess) the first time. Off by default so nothing
  claims trackpad space or prompts unexpectedly at launch.

App is unsandboxed (no App Sandbox entitlement), so the private-framework
dlopen and Core Audio calls are permitted (PRD §4: can't ship on the MAS).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements a system-wide trackpad gesture for controlling system volume by reading raw trackpad contacts via the private MultitouchSupport framework and updating the volume using Core Audio. The feedback focuses on ensuring thread safety and improving gesture precision. Key recommendations include introducing a lock to synchronize access to gesture state variables across threads, adjusting the volume throttling logic to guarantee that boundary volumes (0.0 and 1.0) are reachable, and refactoring the volume address property to a file-scope constant to prevent exclusivity violations.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +33 to +37
private var engaged = false
private var armed = false
private var baselineY: Float = 0
private var startVolume: Float = 0
private var lastAppliedVolume: Float = -1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

To prevent data races on the gesture state variables (engaged, armed, etc.) between the main thread (in disable()) and the background MultitouchSupport callback thread (in handleFrame), we should introduce a lock to synchronize access.

Suggested change
private var engaged = false
private var armed = false
private var baselineY: Float = 0
private var startVolume: Float = 0
private var lastAppliedVolume: Float = -1
private let lock = NSLock()
private var engaged = false
private var armed = false
private var baselineY: Float = 0
private var startVolume: Float = 0
private var lastAppliedVolume: Float = -1

Comment on lines +72 to +73
engaged = false
armed = false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Wrap the state reset in lock.withLock to ensure thread safety when disabling the gesture engine.

        lock.withLock {
            engaged = false
            armed = false
        }

Comment on lines +87 to +127
private func handleFrame(_ touches: UnsafePointer<MTTouch>, _ count: Int) {
// We only care about a clean two-finger contact. Anything else ends the
// gesture, so lifting or adding a finger cleanly disengages.
guard count == 2 else {
engaged = false
armed = false
return
}

let a = touches[0].normalized.position
let b = touches[1].normalized.position
let avgY = (a.y + b.y) / 2

if !engaged {
engaged = true
// Arm only when BOTH fingers land in the left-edge strip — this is
// what separates a deliberate edge slide from ordinary two-finger
// scrolling, which starts anywhere on the surface.
armed = a.x < edgeThreshold && b.x < edgeThreshold
baselineY = avgY
startVolume = volume.currentVolume() ?? 0
lastAppliedVolume = -1
return
}

guard armed else { return }

let delta = avgY - baselineY
guard abs(delta) > deadZone else { return }

// Slide up (y increases) raises volume; one `travelForFullRange` of
// vertical travel sweeps the whole 0...1 range.
let target = startVolume + delta / travelForFullRange
let clamped = min(max(target, 0), 1)

// Throttle Core Audio writes: only when it moves ~1.5% (≈64 steps),
// finer than the 16 hardware increments but without spamming the device.
guard abs(clamped - lastAppliedVolume) >= 0.015 else { return }
lastAppliedVolume = clamped
volume.setVolume(clamped)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This suggestion addresses two important issues in handleFrame:

  1. Thread Safety / Data Race: The gesture state is mutated on the background MultitouchSupport callback thread, but some state is also reset on the main thread in disable(). Wrapping the body in lock.withLock ensures thread-safe access.
  2. Boundary Volume Reachability: The 0.015 (1.5%) throttle threshold can prevent the volume from reaching exactly 0.0 (muted) or 1.0 (100%) if the final gesture increment is small. Adding || clamped == 0 || clamped == 1 ensures these boundary values are always reachable.
    private func handleFrame(_ touches: UnsafePointer<MTTouch>, _ count: Int) {
        lock.withLock {
            // We only care about a clean two-finger contact. Anything else ends the
            // gesture, so lifting or adding a finger cleanly disengages.
            guard count == 2 else {
                engaged = false
                armed = false
                return
            }

            let a = touches[0].normalized.position
            let b = touches[1].normalized.position
            let avgY = (a.y + b.y) / 2

            if !engaged {
                engaged = true
                // Arm only when BOTH fingers land in the left-edge strip — this is
                // what separates a deliberate edge slide from ordinary two-finger
                // scrolling, which starts anywhere on the surface.
                armed = a.x < edgeThreshold && b.x < edgeThreshold
                baselineY = avgY
                startVolume = volume.currentVolume() ?? 0
                lastAppliedVolume = -1
                return
            }

            guard armed else { return }

            let delta = avgY - baselineY
            guard abs(delta) > deadZone else { return }

            // Slide up (y increases) raises volume; one `travelForFullRange` of
            // vertical travel sweeps the whole 0...1 range.
            let target = startVolume + delta / travelForFullRange
            let clamped = min(max(target, 0), 1)

            // Throttle Core Audio writes: only when it moves ~1.5% (≈64 steps),
            // finer than the 16 hardware increments but without spamming the device.
            guard abs(clamped - lastAppliedVolume) >= 0.015 || clamped == 0 || clamped == 1 else { return }
            lastAppliedVolume = clamped
            volume.setVolume(clamped)
        }
    }

Comment on lines 13 to +18
final class VolumeController {
// TODO: Phase 5 — AudioObjectSetPropertyData on
// kAudioHardwareServiceDeviceProperty_VirtualMainVolume, with smoothing.
private var volumeAddress = AudioObjectPropertyAddress(
mSelector: kAudioHardwareServiceDeviceProperty_VirtualMainVolume,
mScope: kAudioDevicePropertyScopeOutput,
mElement: kAudioObjectPropertyElementMain
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since VolumeController is called from the background MultitouchSupport callback thread, having volumeAddress as an instance var and passing it by reference (&volumeAddress) to Core Audio functions can lead to thread-safety and exclusivity violations in Swift. Moving volumeAddress to the file scope as a private let constant makes the class completely stateless and thread-safe.

Suggested change
final class VolumeController {
// TODO: Phase 5 — AudioObjectSetPropertyData on
// kAudioHardwareServiceDeviceProperty_VirtualMainVolume, with smoothing.
private var volumeAddress = AudioObjectPropertyAddress(
mSelector: kAudioHardwareServiceDeviceProperty_VirtualMainVolume,
mScope: kAudioDevicePropertyScopeOutput,
mElement: kAudioObjectPropertyElementMain
)
private let volumeAddress = AudioObjectPropertyAddress(
mSelector: kAudioHardwareServiceDeviceProperty_VirtualMainVolume,
mScope: kAudioDevicePropertyScopeOutput,
mElement: kAudioObjectPropertyElementMain
)
final class VolumeController {

tirthfx and others added 18 commits July 28, 2026 23:06
…kdrop

Folds in the creative polish items and reworks the lock-screen expanded/lyrics
states to match the Apple-Music reference (image 2).

- Album-art accent color: AlbumArtColor samples a vibrant tone from the current
  art (downsample + saturation-weighted average, normalized to always pop);
  MediaMetadataService publishes it as accentColor. It tints the current lyric
  line (both notch and lock screen), the album-art bloom, and the glass rims.
- Liquid-glass clock: frosted material-filled numerals over the blurred
  wallpaper with a soft light bloom; compact at the top in the expanded/lyrics
  states, large/centered at the widget level.
- Blur instead of dim: expanding now frosts the wallpaper (blur + a light tint)
  rather than fading it to near-black, and the lyrics layout places the clock
  small at top with album art + controls on the left and lyrics on the right.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Match the macOS lock-screen reference: much larger heavy frosted clock
(132pt), compact minimal now-playing card (300pt, smaller art/text/
transport, hairline edge instead of the accent rim), and a cosmetic
"Touch ID or Enter Password" hint at the bottom.

Add "Set Lock Screen Wallpaper…" to the menu bar: pick any image as the
lock-screen backdrop (persisted in UserDefaults), with "Use Desktop
Wallpaper" to revert.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LiquidGlassPanel gains a .clear style: untinted .glassEffect(.clear) on
macOS 26 (material-only approximation pre-26). The lock-screen
now-playing card now uses it — fully transparent glass, no dark slab —
with soft text shadows keeping white content legible.

Clock numerals drop from heavy to bold rounded (reference glyphs are
thick, not bubbly) and switch to .thinMaterial with a lighter white
overlay so they read as clearer glass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Clicking the widget now zooms: the expanded art+pill cluster springs
  up from 0.3 scale while the compact card swells away (asymmetric
  transitions replace the inert matched-geometry morph on L1<->L2; the
  L2<->L3 hero morph stays).
- Top-left glass droplet button reveals a slider adjusting the idle
  wallpaper blur (0-60, persisted in UserDefaults).
- Clock switches from rounded to SF Pro Display Bold.
- Expanded control pill width matches the 320pt album cover.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… shadow

- Album art 320 -> 360 with tighter 12pt corners; art-to-pill gap 18.
- Control pill shrunk to a reference-style mini-player: 260pt wide
  (clearly narrower than the art), 14/11pt text, small transport row,
  hairline edge replacing the accent rim; accent bloom on the art
  softened to a whisper (reference shadow is neutral).
- Accent param dropped from the pill (lyrics highlight still uses it).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Control pill switches to the fully transparent LiquidGlassPanel
  (.clear) treatment, matching the compact widget.
- Album cover 360 -> 480 (the reference art fills ~half the screen
  height); pill widened to 300 to stay proportional.
- Clock drops the AM/PM suffix (reference is bare hour:minute) and the
  compact variant grows to 78pt with a 15pt date.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Clock: hardcoded "h:mm" (locale AM/PM omission zero-padded the hour
  to "04:00"; reference is "9:24") and proportional digits — tabular
  figures read too wide and boxy.
- Album cover corners 12 -> 6, matching the reference's near-square
  edges (widget placeholder too).
- Grow-on-click now actually animates: the expanded cluster springs
  from 35% scale via onAppear state inside LockScreenExpandedView. The
  previous .transition approach never fired — the whole switch-case
  subtree swaps, so nested transitions are ignored.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…onstants

Rebuild the lock-screen L1/L2 states per the measured spec:

- New LockScreenLayout enum holds every size/radius/gap/spring constant
  in one place, tagged measured vs estimated (compact numbers came from
  an angled photo and are meant to be nudged on-device).
- L1 compact state is now clock -> small centered art (215pt, r16) ->
  condensed mini pill (160x85, r10), replacing the old horizontal
  widget card. Pill content condenses (smaller type, thinner scrubber,
  small transport) via isCompact; ScrubberView gains a compact variant.
- L1<->L2 is ONE persistent view tree: the same art/pill animate their
  frames, radii, and gaps in a single coordinated spring (response
  0.45, damping 0.75) — no more crossfade, no out-of-sync morph.
  matchedGeometryEffect remains only for the L2<->L3 lyrics hero.
- Clock font size fixed at 96pt in both states; expanding only shifts
  it down 12pt (spec: translateY only, never resizes).
- Tap art or track info toggles expand/collapse (rapid re-taps retarget
  the spring safely); Esc/click-away still step down.
- LockScreenExpandedView deleted — superseded by the cluster.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per notchbeat-lockscreen-fix-prompt.md (pixel-analyzed ratios, scale to
any screen; all in LockScreenLayout):

- Idle no longer rests in the expanded-card layout (the ref-3 bug). It
  now shows the Large Clock (digits ~23.5% of screen height) with a
  small glass chip in the bottom-left corner: album thumbnail + track
  title/artist only — no scrubber or transport at chip size.
- Tapping the chip grows it into the centered card: clock digits scale
  down to 9.8% H (date keeps its size), album cover is a true centered
  square (43.3% H, top edge 33% down), pill 21.3% W x 16.9% H starting
  5.4% H below the art.
- The morph is two persistent views (glass container + album art)
  animating frames/positions/radii in the one spec spring — the cover
  visibly flies out of the chip's thumbnail slot to screen center, so
  the expand reads as growing, never a crossfade or snap. Container
  content (chip line vs full controls) crossfades inside.
- Clock renders at idle size and scales down for the card state, so
  the big resting clock stays crisp.
- LockScreenControlPill reduced to the fixed lyrics-state (L3) pill.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… card

Per notchbeat-lockscreen-fix-round2-prompt.md; animation untouched.

- Idle clock 1.17x (digit block ~21.4% of H); expanded clock 1.48x
  (~10.5% of H) — both high-confidence pixel measurements.
- Idle now-playing: the bottom-left chip becomes a small CENTERED
  mini-player (280x112 at 60% down): thumbnail + title/artist, thin
  scrubber, small transport row — matching the target-3 reference photo
  (its visual layout won over the doc's off-angle "bottom-left" pill
  numbers; confirmed with the user).
- Expanded card: album pulled up (top edge 33% -> 22% of H) and grown
  to 47% of H (~31% of W); pill width now matches the art exactly and
  slides down under the repositioned cover, same gap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per notchbeat-lockscreen-fix-round3-prompt.md; animation untouched.

- Idle clock font ratio 0.275 -> 0.34: digit INK is ~63% of font size,
  so the previous ratio rendered 17.3% ink against the 21.4% target.
  Expanded clock keeps its correct absolute size (ratio math preserved).
- Idle now-playing reshaped: the centered mini-player becomes a compact
  left-aligned chip (240x64 at 53% down, 28pt left margin) holding only
  the album thumbnail + title/artist — scrubber and transport removed
  from idle entirely; they belong to the expanded pill.
- Expanded pill height ratio 0.169 -> 0.21 (+30%, read cramped);
  clock/album untouched (measured correct).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per notchbeat-lockscreen-fix-round4-prompt.md (absolute targets, not
deltas). Album/pill morph animation untouched.

- Clock ratios corrected against SF Pro's real cap height (~71% of font
  size, not 63%): idle font 0.30 -> 21.4% ink (round-3's 0.34 overshot
  to mid-20s), card font 0.147 -> 10.5% ink.
- Idle chip locked to the absolute spec: 18% W x 14% H (ratio-based
  now), 56pt thumb, 14/12pt text; still left-aligned at 53% down,
  thumbnail + title/artist only.
- Expanded pill: system full-intensity Liquid Glass (.regular Glass,
  new LiquidGlassStyle case) instead of clear — real blur/vibrancy —
  plus a specular top-edge rim (white gradient hairline) on the
  container in both states. Expanded album/pill geometry confirmed
  in-spec, unchanged.
- Clock size change now SNAPS between states (transaction strips the
  spring); the cover/pill morph keeps animating as before.

Duplicate-widget report investigated: the idle tree renders exactly one
now-playing element — the "stray mid-screen thumbnail" matches the
morphing art view captured mid-flight during collapse, not a leftover
instance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reverts 7e3270a, ce3a247, b02d699 — back to the round-3 state
(bigger idle clock, minimal left-aligned idle chip, taller pill).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant