[00139] Implement Rust Widget APIs for Chat, Audio, Camera, and Signature Inputs

[rorychatt]

Fixes #127

00139 — Implement Rust Widget APIs for Chat, Audio, Camera, and Signature Inputs

Seven Rust widgets for React components that already shipped in the vendored
frontend but had no Rust producer (issue #127). No file under src/frontend
changed.

Branch tendril/00139-ImplementRustWidgetAPIsForChatAudioCameraAndSignatureInputs,
four commits off origin/main at c528820:

Commit	Subject
f95602d	Add Chat and media capture widgets to the rusty crate
9242799	Exercise the seven new widgets through the E2E harness
f9d8dd9	Document the chat and media capture widgets
7c0f43e	Canonicalize the PascalCase event names Ivy actually sends

Changes

Four chat widgets (rusty/src/widgets/chat.rs, new). Chat is the thread
plus composer, holding ChatMessage children, a placeholder, a streaming
flag, quick replies, size and density, and the on_send(String) / on_cancel()
events. ChatMessage carries a ChatSender (User / Assistant) and
arbitrary children, with ChatMessage::user(..) / ::assistant(..)
shorthands. ChatLoading is a typing indicator with no properties at all;
ChatStatus is one line of text. All four use #[derive(Widget)] — no
hand-written to_json was needed.

Three media inputs (rusty/src/widgets/media_inputs.rs, new). AudioInput
(microphone), CameraInput (stills or clips) and SignatureInput (pointer-drawn
signature) each deliver their result as a base64 data URL. Two transports coexist
deliberately: on_capture / on_sign send the data URL over the event socket
(the Rust-native path, and the one the harness exercises), while upload_url
matches what the React widgets do — POST the blob themselves and fire no capture
event. Rusty has no upload endpoint today, so upload_url is a pass-through prop
for a future adapter; the module doc says so.

Five new event names (rusty/src/core/event_registry.rs). Send, Cancel,
Capture, Sign, Clear joined EventName, so canonicalize resolves the
browser-shaped spelling to the lowercase name the derive registers under.

A canonicalization fix in the same file. EventName::normalize stripped only
a lowercase on prefix, so from_str("OnSend") returned None and dispatch
looked the handler up under "OnSend" while it was registered as "send" — the
handler never fired. On is exactly the casing the real client sends
(ChatWidget.tsx calls eventHandler("OnSend", ..), ButtonWidget.tsx
("OnClick", ..)), and ivy_node.rs's module doc already claimed canonicalize
"accepts OnClick, onClick and click alike". It now does. This is wider than
this plan's own widgets — it repairs inbound Ivy events for every pre-existing
widget too. See recommendations.md.

Ivy mapping. Seven mechanical snake_case → Ivy.PascalCase arms in
widget_names.rs (chat → Ivy.Chat, and so on), with the inventory counts and
the every_widget_type_is_mapped scan floor raised 38 → 45 so the guard stays
tight. In ivy_node.rs, IVY_EVENT_NAMES gained "OnCancel" and "OnSend"
(the two ChatWidget.tsx reads) and ENUM_PROPS gained "sender" so
"user" / "assistant" title-case to what ChatMessageWidgetProps declares.
facingMode is deliberately not in ENUM_PROPS: that value goes straight to
getUserMedia, which accepts lowercase only.

Harness coverage. Four apps in rusty-server/src/bin/widget_harness.rs and
seven case arms in the client renderer, added identically to both copies of it
(e2e/app/index.html and rusty-desktop/assets/index.html) — there is no
automated parity check between them, so the desktop shell would otherwise render
[Unknown widget: chat]. The arms are plain DOM with data-* hooks and fixed
stub payloads: no getUserMedia, no MediaRecorder, no device permissions
anywhere.

Docs. New pages 03_widgets/34_chat.md and 03_widgets/35_media_inputs.md,
plus the counts in 02_concepts/02_widgets.md (38 → 45 types, 25 → 32
mechanical mappings).

API Changes

New, all re-exported from rusty::widgets and therefore from rusty::prelude:

Type	Wire type	Ivy component
Chat	chat	Ivy.Chat
ChatMessage	chat_message	Ivy.ChatMessage
ChatLoading	chat_loading	Ivy.ChatLoading
ChatStatus	chat_status	Ivy.ChatStatus
AudioInput	audio_input	Ivy.AudioInput
CameraInput	camera_input	Ivy.CameraInput
SignatureInput	signature_input	Ivy.SignatureInput

Supporting enums: ChatSender { User, Assistant }, FacingMode { User, Environment }, CaptureMode { Image, Video } — all Default, all serializing
camelCase.

Chat::new()
    .message(ChatMessage::user(TextBlock::paragraph("What is Rusty?")))
    .message(ChatMessage::assistant(ChatLoading::new()))
    .placeholder("Ask something…")
    .quick_reply("Summarise this")
    .streaming(true)
    .on_send(move |text| history.update(|h| { let mut n = h.clone(); n.push(text.clone()); n }))
    .on_cancel(|| println!("interrupted"))

AudioInput::new().label("Record a note").on_capture(|url| println!("{}", url.len()))
CameraInput::new().facing_mode(FacingMode::Environment)
SignatureInput::new().placeholder("Sign here").pen_thickness(2.0).on_sign(move |png| sig.set(png))

EventName gained five variants — it is a public enum, so a downstream
exhaustive match on it would need the new arms.

Behaviour change: EventName::from_str / canonicalize now accept the On
prefix ("OnClick", "OnSend"). Strictly wider acceptance; nothing that parsed
before parses differently, and "online" / "Online" are still not read as
on + line.

Nothing was removed or renamed. No rusty-ivyml change: its element table is a
deliberate allowlist, not an inventory.

Files Modified

New:

• rusty/src/widgets/chat.rs (+518)
• rusty/src/widgets/media_inputs.rs (+930)
• e2e/tests/widgets/chat.spec.ts (+82)
• e2e/tests/widgets/media-inputs.spec.ts (+130)
• rusty-docs/docs/03_widgets/34_chat.md (+98)
• rusty-docs/docs/03_widgets/35_media_inputs.md (+128)

Changed:

• rusty/src/widgets/mod.rs (+4) — modules and re-exports
• rusty/src/core/event_registry.rs (+57/−1) — five event names, On prefix
• rusty/src/shared/widget_names.rs (+41/−6) — seven Ivy arms, counts, scan floor
• rusty/src/shared/ivy_node.rs (+77/−3) — IVY_EVENT_NAMES, ENUM_PROPS, docs
• rusty-server/src/bin/widget_harness.rs (+179) — four harness apps
• e2e/app/index.html (+169) — seven renderer arms
• rusty-desktop/assets/index.html (+169) — the same seven, byte-identical
• rusty-docs/docs/02_concepts/02_widgets.md (+2/−2) — counts

Manual Testing

Gates, all from AGENTS.md, all re-run after the final commit:

Gate	Result
cargo fmt --all -- --check	exit 0
cargo clippy --workspace --all-targets --no-default-features -- -D warnings	exit 0, no warnings
cargo clippy -p rusty-desktop --all-targets -- -D warnings	exit 0, no warnings
cargo build --workspace --no-default-features	exit 0, no warnings
cargo build -p rusty-desktop	exit 0, no warnings
cargo test --workspace --no-default-features	774 passed, 0 failed, 4 pre-existing ignored
node scripts/check-harness-script.js	exit 0, "inline <script> parses (1067 lines)"
npx playwright test (from e2e/)	158 passed (143 pre-existing + 15 new)
pnpm install --frozen-lockfile && pnpm run build (from src/frontend)	exit 0

The 15 new Playwright tests drive the real harness binary end to end: a chat
send round trip appends a bubble the server owns, a quick reply arrives as the
same event, cancel increments a server-side counter, and each media input's stub
capture arrives as a data URL. cargo fmt needs a prior cargo build in a fresh
worktree, because rusty-docs/src/generated/ is build-script output and
gitignored.

Two things worth knowing for anyone re-running this:

• One E2E test failed on the first full run. AudioInputApp stored focus as a
  bool; setting it triggers a rebuild that replaces the focused button, whose
  blur reset the flag before the assertion could see it. Fixed by counting
  focuses and blurs instead, with a comment in the harness explaining why.
• npm ci / pnpm install fail on this machine with
  UNABLE_TO_GET_ISSUER_CERT_LOCALLY. Worked around outside the repo with
  NODE_EXTRA_CA_CERTS pointing at a dump of the macOS system trust roots; no
  repo file was changed for it.

Not tested: no browser ever touched a microphone or camera, and no signature was
drawn with a pointer — the renderer arms send fixed stub payloads. The Ivy React
components themselves are exercised by nothing here; what is proven is the wire
shape Rusty emits and that events come back. cargo run -p rusty-desktop was
not opened by hand.

---

Commits

• 7c0f43e [00139] Canonicalize the PascalCase event names Ivy actually sends
• f9d8dd9 [00139] Document the chat and media capture widgets
• 9242799 [00139] Exercise the seven new widgets through the E2E harness
• f95602d [00139] Add Chat and media capture widgets to the rusty crate

---

Created using Ivy Tendril.