Skip to content

[00140] Add use_stream and use_upload Hooks With Server Side Multipart Upload - #134

Merged
rorychatt merged 6 commits into
mainfrom
tendril/00140-AddUsestreamAndUseuploadHooksWithServerSideMultipartUpload
Aug 10, 2026
Merged

[00140] Add use_stream and use_upload Hooks With Server Side Multipart Upload#134
rorychatt merged 6 commits into
mainfrom
tendril/00140-AddUsestreamAndUseuploadHooksWithServerSideMultipartUpload

Conversation

@rorychatt

Copy link
Copy Markdown
Contributor

Fixes #120

00140 — Add use_stream and use_upload Hooks With Server Side Multipart Upload

Plan 00140 (job 00687, issue #120) is implemented on branch
tendril/00140-AddUsestreamAndUseuploadHooksWithServerSideMultipartUpload in
six commits. All nine verifications are resolved: seven Pass, two Skipped (the two
frontend-only ones — this plan changed no frontend file, by design).

What was built

A stream hook. use_stream drives a futures::Stream into view state chunk by
chunk on a spawned task; use_stream_text is the LLM-token convenience that
concatenates instead of collecting. Both carry a StreamStatus
(Idle/Streaming/Done/Error), an optional retry budget with a delay, an
optional cap on retention, auto_start: false for streams a user starts, and
restart()/stop() closures safe to call from an event handler.

An upload hook and the server half it needs. use_upload registers a slot on
mount and publishes its URL; use_upload_to hands each completed file to a sink
instead of holding it in view state. Behind them, UploadService is a
per-connection registry with the same drop-to-unregister design as
DownloadService, and upload_handler serves
POST /rusty/upload/{connection_id}/{upload_id} by reading the multipart body chunk
by chunk — which is what makes progress reporting, mid-flight cancellation and
rejecting an oversize body without buffering it all possible.

No frontend change, deliberately. The client half already shipped:
uploadFileWithProgress POSTs a FormData with one field named file and treats
200..300 as success. The endpoint was written to that contract exactly. The Rust
FileInput widget that would carry an uploadUrl to the browser is issue #128 and
out of scope here.

Commits

SHA Subject
a6e686f Enable axum's multipart feature for server side uploads
f0988df Add the UploadService registry and register it per connection
9108d81 Add the use_stream and use_stream_text hooks
13849c6 Add the use_upload and use_upload_to hooks
590bafc Serve multipart uploads on /rusty/upload/{connection}/{upload}
23f8639 Document the use_stream and use_upload hooks

Ordered so each commit builds on the last: the feature flag, then the registry, then
each hook with its own exports, then the endpoint (whose tests need the hook), then
the docs. The two hook commits each carry only their own lines of the three shared
export files (hooks/mod.rs, lib.rs, hook_rules.rs), which needed a separate
cargo fmt pass per intermediate state because a partial use list rewraps
differently from the final one.

Verifications

Verification Result
RustFmt Pass
RustClippy Pass
VitePlusCheck Skipped — no frontend source changed
RustBuild Pass
NpmBuild Pass
RustTest Pass — 788 passed, 0 failed, 5 pre-existing ignored
VitePlusTest Skipped — no frontend source changed
PlaywrightE2E Pass — 143 passed
CheckResult Pass

55 net new tests (check-test-inventory.sh: 741 → 796): 19 for use_stream, 14 for
use_upload, 14 for UploadService, 7 endpoint tests over a real loopback socket,
plus two extended session.rs tests. Every server-binding test uses 127.0.0.1:0.

Three deviations from the plan, and why

1. UploadError::is_client_error() was implemented as status_code(). The
plan's name reads as a predicate, but its signature and mapping are a converter. A
bool-looking method that returns StatusCode is a trap at every call site. The
mapping is exactly what the plan specifies: TooLarge → 413,
TooSmall/RejectedMimeType → 415, NoFile/Cancelled/Transport → 400.

2. The early oversize rejection compares against max_bytes + MULTIPART_ENVELOPE_ALLOWANCE (8 KiB), not max_bytes. The plan's
"Content-Length vs max_bytes → 413 early" is unsound as written:
Content-Length covers the boundary lines, the part headers and the file name as
well as the bytes, so it is only ever an upper bound on the file's own size. A
bare total > max_bytes would reject a file of exactly max_bytes with a 413 —
a limit that lies about itself. The allowance keeps the early rejection for the case
it exists for, a body far too big to be worth reading, while the exact limit is
still enforced chunk by chunk as the body arrives. The constant carries this
reasoning in its doc comment, and both paths have their own test.

3. add_upload_with_cancel was added alongside add_upload. A view must be
able to cancel a body that is already arriving, but the handle only exists after the
mount effect has run — so the cancel flag has to be owned by the hook's use_ref
and handed to the service, not allocated by it. add_upload is unchanged for
callers that do not need this. Both are tested.

Two smaller calls, recorded so they are not mistaken for accidents:

  • restart() also resets the status to Idle. The plan does not mention it, but a
    restarted stream that still reports Done is wrong for any consumer keying off
    status — and it was making two tests pass vacuously.
  • use_stream_text's max_chunks caps how many chunks are appended, rather than
    ring-buffering, because a concatenated String has no meaningful "drop the oldest
    chunk". use_stream uses a true ring cap, as specified.

The one thing worth knowing if you touch these tests

The plan suggested testing retry_delay with tokio::time::pause + advance. That
works only for a test that advances the clock itself. Under start_paused, tokio
auto-advances to the nearest timer, and a polling helper's own 5 ms sleeps are
always nearer than a multi-second retry delay — so two retry tests spun through
2 s of virtual time and timed out without the retry ever firing. Those two now run
on the real clock with a 10 ms delay, and a dedicated start_paused test
(test_a_retry_waits_out_the_retry_delay) drives the clock explicitly to prove the
delay is honoured: no retry at 9 s of a 10 s delay, retry by 11 s.


  • a6e686f [00140] Enable axum's multipart feature for server side uploads
  • f0988df [00140] Add the UploadService registry and register it per connection
  • 9108d81 [00140] Add the use_stream and use_stream_text hooks
  • 13849c6 [00140] Add the use_upload and use_upload_to hooks
  • 590bafc [00140] Serve multipart uploads on /rusty/upload/{connection}/{upload}
  • 23f8639 [00140] Document the use_stream and use_upload hooks

Created using Ivy Tendril.

The Multipart extractor the upload endpoint needs is behind a feature flag.
Workspace-wide because rusty-server and rusty-desktop share the dependency.
Slots are keyed by connection so an upload URL only resolves for the session
that created it, and dropping the handle a view holds unregisters the slot.
The cancel flag is shared with the caller so a view can cancel a body that is
already arriving, before the handle exists.
Drives a futures::Stream into view state chunk by chunk on a spawned task,
with an optional retry budget and a ring cap on how much is retained. restart()
bumps a generation State so the effect re-registers, and both restart and the
effect cleanup abort the previous task before anything new writes the state.
Registers a slot on mount and publishes its URL, so a view renders its picker
only once the URL exists. Progress is what the server has received, capped at
99 until the bytes are in hand. use_upload_to hands each file to a sink instead
of holding it in view state, and reports a sink error the same way a rejected
MIME type is reported. Both hooks use the same slot layout so swapping one for
the other does not shift any later hook.
Reads the file field chunk by chunk rather than through Field::bytes(), which
is what makes progress reporting, mid-flight cancellation and rejecting an
oversize body without buffering it possible. Every failure past a resolved slot
reports itself through UploadEvent::Failed first, because the browser only sees
the status code. The raised body limit is a layer on this route alone.
@rorychatt rorychatt self-assigned this Aug 10, 2026
@rorychatt
rorychatt merged commit 7e13e14 into main Aug 10, 2026
@rorychatt
rorychatt deleted the tendril/00140-AddUsestreamAndUseuploadHooksWithServerSideMultipartUpload branch August 10, 2026 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Hooks] Implement use_stream (LLM/Data Streaming) and use_upload (Multipart File Upload) Hooks

1 participant