Fleet lab: swarm coordination sandbox + teaching tool#2
Open
publu wants to merge 121 commits into
Open
Conversation
…elf-deadlock - rosbridge.py: a websocket recv timeout on a quiet socket (a sim robot with no /tf chatter) was treated as a disconnect, causing an infinite reconnect loop that never replays subscriptions. Timeouts now continue. - ros_camera.py: state() called is_active() while already holding the same non-reentrant lock, freezing every behavior's see() on first use. The freshness check is now inlined.
…oes live The arena/deck pages already degrade gracefully on a static host, but silently: relative /api fetches 404 and the page stays in demo mode even when a live roborun is one port away. runtime-base.js (loaded first by both pages) wraps fetch: /api calls resolve same-origin first, then probe http://127.0.0.1:8765. A badge shows the mode; in demo mode it keeps probing, so starting roborun upgrades the open page to the live cockpit with one click. Server side: Access-Control-Allow-Origin on all responses (deduped out of do_OPTIONS — doubled CORS headers are rejected by browsers) plus Access-Control-Allow-Private-Network for Chrome PNA preflights.
The whole thesis is switch from sim to robot without changes — so the arena, not a separate deck, is what a connected robot looks like. When /api/ros/health reports a robot, the same arena page enters robot mode: - pose/heading/altitude come from the telemetry handle (SIM_SPEC contract) and drive the same bot body; the level hides, the accumulated lidar cloud is the map, the minimap and telemetry panels read as before - an EYES panel shows /api/camera/stream — the same pixels robot.see() runs YOLO on - WASD publishes real cmd_vel through /api/ros/move (now with linear_z for drones); behaviors keep running server-side, untouched - pushState is gated off: feeding the arena backend while a robot is connected would flip get_arena().is_active() and silently reroute robot.see()/move() from hardware to the browser sim Plus two host-fallback fixes with the same root cause: _get_ros_client and RosTelemetry._try_subscribe demanded a profile robotIp and went dead without one, even while a live connection existed — both now ride the already-connected client. This was why robot_type never resolved (and why drone cmd_vel fell back to /cmd_vel).
The camera stream was last-writer-wins: webcam and robot camera both wrote /tmp/roborun_frame.jpg, so the EYES panel showed your desk while claiming to be the robot. Each pipeline now writes its own file and /api/camera/stream takes ?source=robot|webcam|auto (auto: fresh robot frames outrank the webcam). New sources layer answers "what can see and what can move, right now": - GET /api/sources — webcam on/off, connected robot + camera state, and rosbridges discovered on the local /24 (plain TCP probe of :9090, cached 60s); POST /api/sources/scan forces a rescan - arena EYES panel gets a source picker (robot camera / webcam) - in sim modes the arena shows a chip for any rosbridge found on the network — one click connects and reloads into robot mode. A robot on your wifi is a source, not a config step.
A connected robot deserves a robot cockpit, not a game with a robot in it. The screenshot version showed DOG—SANDBOX missions, practice RUNS, sim crates in the main view, and a sandbox policy editor one click away from commanding live hardware — the exact path by which player_policy (forward=0.8 at 10 Hz) flew the test drone to 53 m and 45 m off the map. In robot mode: - MISSION/LEVELS/RUNS panels and their toolbar buttons hide (CSS via body.robot-mode); loadLevel can no longer resurrect the sim level - the POLICY panel becomes the robot's behavior editor: it loads the source of the behavior actually running (new POST /api/behaviors/read, stem-only, no paths), RUN hot-reloads that file, STOP disables it - deploying to hardware is deliberate: RUN asks for confirmation and names the file; the LLM mission compiler gets a be-conservative context instead of the level brief - footer says what WASD really does now: drives the real robot - room/practice telemetry hides; main camera defaults to chase
…hero
The arena was still a game with a robot crudely piped in — a low-poly dog
floating in a void, the real camera shrunk to a corner, sim crates in
view, the onboarding splash ('pick a robot, pick a task') greeting a live
drone. A connected robot now gets a purpose-built cockpit instead:
- the robot camera fills the stage (full-bleed, cinematic vignette) with
live YOLO detection boxes overlaid (new /api/robot/detections, normalized)
- a glass identity bar: type glyph, LIVE pulse, host, and ALT/SPEED/HDG
telemetry chips; OSD reticle + POS/ALT/HDG readouts over the feed
- a tactical minimap (range rings, trail, heading wedge; lidar when present)
- POLICY slides in to edit the *running* behavior; DEPLOY is confirmed
- an event ticker of the robot's live decisions
- the game panels, 3D arena, toolbar, and splash are fully suppressed in
robot mode; the splash is gated so it never shows over a robot
Camera served as single JPEG frames (/api/camera/frame) polled by the
client — deterministic, unlike an MJPEG <img> that half-paints.
Telemetry hardened against flaky rosapi discovery: /rosapi/topics times
out on this setup, which left type=webcam_only and no pose. Now when
discovery returns empty, trust the type roborun connect saved and
subscribe to the standard topics blind (a subscribe to a not-yet-seen
topic is harmless and flows when it appears).
Answering 'I don't want the drone — give me something else even though ROS is connected', and making the cockpit a complete shell: - SOURCE picker: every robot/sim this runtime can reach in one menu — the connected robot (live), the browser sim arena, and any rosbridge found on the LAN. Pick the sim and the page pins to it (localStorage) without disconnecting the robot; a chip offers the way back. Multiple ROS robots just appear. - the policy editor is syntax-highlighted now (a colored <pre> underlay behind a transparent textarea — keywords, defs, strings, decorators, comments, numbers) instead of plain white-on-black. - TIMELINE panel (bottom-left, mirroring the tactical map) streams the robot's decisions and sightings with timestamps and source-colored dots — the same stream+map→timeline surface the sim arena has, so the experience is consistent whatever the source is. - DECK link moves into the top bar. The cockpit shell — camera stream (hero), tactical map (objects, moving vs stationary, range rings), timeline, policy — is now one consistent UX; what fills it is the source.
…urce Pablo's vision: the sim IS the robot's visual view — what the camera would see — so there should be one view, not a game UI and a robot UI. The stream, point map, and timeline are all derived from inputs (camera, cloud, pose) that both a sim and a ROS robot provide. The cockpit is now the universal shell, generalized to enterCockpit(src): - src=robot: the stream is the robot camera (frame-polled); map + telemetry from the ROS endpoints. - src=sim: the stream IS the 3D arena render (POV camera, full-screen behind the chrome); telemetry from the sim body; the tactical map from the sim's world-located sightings + trail; DEPLOY/HOLD run the same policy through the game's path; a LEVELS button (sim-only) picks robot + task. Both render the same HUD, tactical map (objects, range rings, moving vs stationary), timeline (decisions + sightings), and policy editor — only the source behind them changes. The game's panel-salad layout and the auto-splash are retired; LEVELS reopens the picker on demand. body.cockpit + .src-robot/ .src-sim gate the source-specific bits.
…m view Three issues from using it live: 1. The sim's timeline showed 'frame … · person' — the connected drone's camera events. The sim cockpit was reading the shared server event log. The sim now keeps its OWN client-side log (simLog): YOLO sightings and policy decisions from its own perception, never the server's. The tactical map likewise builds objects from the sim's client raycast detections (currentDets), not the contaminated server sightings. 2. The sim 'rotated like an idiot' — POV put you inside the dog's head as its policy turned. The sim hero is now the chase camera: a stable 3rd-person view of the robot in its world. 3. The policy panel's header and close were hidden behind the top bar and it read as an inaccessible wall of code. It's now a framed floating panel below the bar — visible BEHAVIOR header, close button, and the DEPLOY/STOP bar. The track readout moved to a centered bottom pill so it no longer collides with the timeline.
…deck There were three doors to two UIs: / and /deck served the legacy flight deck, /arena served the cockpit, and each linked to the other. Confusing and redundant. Now /, /deck and /arena all serve the cockpit — the single view. A connected robot shows its camera/map/timeline; no robot shows the sim; the SOURCE picker switches between them. The DECK button and cross-links are gone. deck.html/deck.js stay in the tree but are no longer routed (the flight recorder still runs server-side; its UI can fold into the cockpit later if needed).
Serving the same page at three paths still read as three things. Now '/' is the one canonical URL for the cockpit; /deck and /arena redirect there.
… editor From live use: - the 'enter cockpit' chip overlapped the top-bar actions (SOURCE/POLICY/ HOLD) — moved it below the bar so nothing is blocked. - the sim immediately span a dog in circles: the starter policy auto-ran. The sim now starts PAUSED (simArmed=false → policy holds); DEPLOY arms it, STOP/HOLD pauses. Calm on arrival, you choose when it moves. - the source picker was a cramped dropdown — now a centered modal with a backdrop and large source cards (Esc / backdrop / ✕ to close). - the policy editor clipped long lines and was too narrow to code in — added an expand toggle (⤢) that widens it to 80vw with a bigger font.
Deploying to a real robot popped the browser's native confirm ('127.0.0.1
says…') — off-brand and ugly. Now it's an in-app modal matching the cockpit:
amber-bordered glass on a dimmed backdrop, a clear warning, Cancel + green
DEPLOY buttons (Esc/backdrop cancels).
- LEVELS did nothing: cockpit CSS force-hid #start. Now #start.show shows in cockpit mode, so the robot+task picker (quadruped/humanoid/drone) opens. - the sim's lidar wasn't on the map and the 3D cloud sprayed the scene. The 3D cloud is now hidden in the cockpit; the sim's 36-ray lidar accumulates into the 2D tactical map (world frame) — the generated map building up. - the cryptic HOLD button is now a clear PAUSE/RESUME toggle whose label reflects state; PAUSE holds the policy, RESUME runs it. DEPLOY/STOP keep it in sync.
Picking a new robot (quadruped/humanoid/drone) from LEVELS rebuilt the sim but the cockpit identity stayed 'DOG'. pollSimCockpit now refreshes the type + glyph each tick, so the header reflects the robot you chose.
Two real bugs the audit surfaced:
1. /api/arena/state 500'd on every push when a recording was active: the
handler reassigned 'h = pose.get("heading")', shadowing the HTTP handler
'h', so the closing send_json(h, ...) got a float. Renamed to 'hd'.
2. The sim's player_policy is a server behavior. While a sim browser feeds
/api/arena/state the arena is active and robot.move() drives the arena dog
(correct). But once that browser closes, the arena goes inactive and the
still-enabled player_policy falls through to drive the REAL robot. Now the
sim disables player_policy on beforeunload (keepalive fetch) and the robot
cockpit disables it on entry — so a closed/abandoned sim can't fly the
drone.
roborun autostarted the webcam (with its privacy light) on every boot to avoid a blank first screen. But if a robot is connected, the robot's camera is the source and the webcam is just an unwanted light on the user's machine. Autostart now skips the webcam when roborun connect has saved a robot; it's still available as a manual camera source.
The static site (and any non-runtime page) now opens on the ROBORUN ARENA menu — the front door to the whole system — instead of dropping straight into a sim. A robot wired directly into THIS runtime still boots to its cockpit; everywhere else the menu leads. The menu gains a fourth card, ROS ROBOT — REAL HARDWARE, alongside the three sim robots. It shows the complete system whether or not anything is connected: a live robot (enter its cockpit), any rosbridge found on the LAN (one-click connect), and an IP field to connect by hand. So 'the same code drives a real robot' is a thing you can actually click, not just a tagline. Picks: a sim robot+task enters the sim cockpit; a ROS robot connects and enters the robot cockpit.
Gazebo and Isaac Sim aren't hardware but speak ROS just the same, so the real split is browser-sim vs anything-on-rosbridge — not sim vs real. The card now reads ROS · GAZEBO · ISAAC · HARDWARE: 'connect anything on rosbridge — a Gazebo or Isaac sim, or a real robot.'
The LAN scan did a plain TCP open of :9090, so any unrelated service on that port showed up as a 'robot to connect to' — confusing false positives. It now completes a websocket handshake (what rosbridge actually is); non-websocket :9090 services are filtered out. Result: only real rosbridge endpoints appear. Also: a connected source with no detectable robot type now reads ROSBRIDGE, not the ugly WEBCAM_ONLY.
The chip is the sim cockpit's affordance to jump back to a connected robot, but pollNetworkRobots returned early in robot mode without hiding it — so it leaked into the robot cockpit, telling you to 'enter cockpit' while you were already in it. It now shows only in the sim cockpit and stays hidden in the menu and robot cockpit (which have the ROS card / are already there).
A rover that drives on /cmd_vel and senses with a laser scanner — no joints, no mavros — was falling through to webcam_only. It's now recognized as a mobile ground robot (quadruped profile: cmd_vel control + lidar + camera), so its pose, lidar map and camera all light up the cockpit instead of reading as 'WEBCAM ONLY'.
The cockpit map drew accumulated lidar as a scatter of 1.5px dots — noisy and hard to read. It now bins returns into world cells and fills them (denser hits = brighter = more confidently a wall), so the swept area reads as solid structure — the built-up map the old deck's ROBOT MAP showed. Also enlarged the panel (280px, taller canvas) since the map carries real information.
You preferred the old deck's arrangeable multi-panel layout (and its ROBOT MAP) over the camera-hero cockpit. So the connected robot now drives the same deck the sim uses, with real telemetry behind every panel: - pose places the bot in the 3D scene; the VIEW panels render it from any angle (top/chase/orbit), and WASD still grabs the wheel - lidar feeds integrateLidar -> the occupancy ROBOT MAP and the 3D point cloud (the map builds up as it drives) - a new EYES panel shows the robot camera with YOLO boxes (robot mode only) - STATUS shows pose/odometer/cmd; POLICY loads and RUN deploys the robot's own behavior (confirmed); MISSION becomes the robot identity - sim-only bits (LEVELS, RUNS) hide in robot mode Same deck, two sources. The camera-hero cockpit is retired (code dormant); enterSimCockpit is now a no-op and the sim runs in the deck directly.
A multi-quadruped "/fleet" sandbox reachable from the robot picker, built to teach how a swarm covers ground over an imperfect radio. - Coverage sim with 4 strategies (lone wolves, gossip, claim-and-yield, one commander) under real limits: radio range, airtime, onboard memory, inbox depth, one goal at a time, lossy links. - Hover any robot to see what it knows: tiles it sensed firsthand (filled) vs. only heard from peers (outlined), with a live knowledge card. - Base station (the "main server") + data points: discover data and relay it home with greedy geographic routing (multi-hop / data-mule). - Environments: open / scattered obstacles / building with LOS-blocking walls. - CONCEPTS panel (expandable) explaining delivered vs dropped, overlap, optimisation, libp2p/gossipsub, stigmergy — plus tooltips on every metric. - "Your algorithm": live JS strategy editor + a Generate button that asks the local LLM to draft a policy (POST /api/fleet/strategy), runnable on the spot. - roborun/swarm/: the same model + strategies + base relay as runnable Python (python -m roborun.swarm), the headless twin; ships with the package. Also: - ROS card "Allow network scan to load robots" button — trips the browser's local-network permission and lists every rosbridge robot found as its own view. - ROS-connected robots reuse the exact sim deck; EYES camera docks into the layout instead of floating. - Replaced the last native confirm() (deploy-to-robot) with the styled modal. - vercel.json (cleanUrls) so /fleet resolves on the static deploy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A freshly opened page (even when a robot is connected on the local runtime) now lands on the picker instead of auto-entering the robot deck — the menu is the front door. Auto-enter only when a robot is explicitly pinned. In the deck, keep a visible '⊞ MENU' button so there's always a way back to select something. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The ROS-parity change made EYES a layout-managed panel, so once the robot deck opened it the saved layout carried it into later sim sessions — where there's no robot camera, leaving it black. Guard it in applyLayout: the EYES panel and its toolbar toggle only show when MODE === 'robot'. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each robot now tracks everKnown (lifetime, never evicted) alongside its memory-capped current set. Hover splits it into IN MEMORY NOW (sensed vs heard, out of the memory cap) and EVER KNOWN (cells discovered, and how many it has forgotten because memory filled). The map overlay adds a faint third tier for the forgotten footprint, under the bright in-memory cells. Mirrored in roborun/swarm (Robot.ever_known + forgotten()). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…l, record-everything channels, headless sim, SimBackend/rl_env/gz, retention, perf benches Implements the buildable items across the six private build specs, each with tests (167 passing): - scenario_defs: seed + run_matrix (A/B+sweeps) + regression_gate - routes/scenarios.py + web/scenarios.html: the Scenarios board (suites, pass-rate, runs) - recorder.close: seal now, anchor async (seal 9.9ms vs up to 30s) - spatial_memory.recall(by=clip|label|near|time): unified retrieval - recorder /cmd /telemetry /gps /cloud channels + post-clamp tap in behaviors.move - simulator headless render gate (full SPS); retention.py GC - sim_backend.py (sense MjData via the handle), rl_env.py (reset/step/obs), gz.py - r2sync list_keys(start_after) cursor; scripts/bench_seal.py + bench_clip.py
- ros_telemetry subscribes /gps/fix,/fix,/navsat/fix (NavSatFix) and folds GPS into the run's MCAP via write_gps + pushes to the telemetry bus. - RunRecorder.channels() exposes written topics; record/stop captures them before close and finalize() writes the channel list into the run manifest, so the browser/search know a run's modalities without parsing the tape.
- isaac.py mirrors gz.py: detect_world (live-stage discovery via the ROS bridge), IsaacClock (lockstep pause/step/reset for determinism), level_to_prims (RoboRun level -> USD stage prims, pure/testable), IsaacRunner (degrades to a dry-run plan offline). Isaac is now 'available' (real driver), not 'planned'. - backends.py probes isaac.detect_world like gz; capability matrix gains photoreal. - tests: registry has 6 backends with valid status; SimBackend satisfies the handle contract; gz+isaac degrade cleanly offline with deterministic plans.
- spatial.py: transform_point (sensor-local -> env world frame), sensor_pose (fixed-camera extrinsics vs robot pose), camera_frustum (view cone geometry), cluster_tracks (merge same-label detections within radius into env-frame object tracks with running-mean centroid + count + first/last seen). - SpatialMemoryStore.object_tracks(): tracks scoped to the active project index. - /api/spatial: the environment's spatial picture — object tracks + registered camera placements/frusta in one frame. - /browser draws a top-down spatial map: tracked objects (sized by count) + camera frusta, auto-scaled to the env frame. Detections + cameras + lidar now read as one picture of 'what is where'. 5 tests.
/run is now a true multi-panel telemetry viewer: play/pause + scrub drive a shared playhead across the time-series panels (velocity, clearance), the camera frame, and a synced event log that highlights + scrolls to the event at the current time. Every panel moves on one clock (spec 02 P2).
…tors - worlds.py: a multi-floor warehouse generator — grid of rooms per floor, aisle walls with doorways, detectable items, and ELEVATORS that stack adjacent floors at a shared shaft. Deterministic by seed. /api/worlds/warehouse. - /fleet-sim + fleet-sim.js: a real N-robot sim (1-40 robots) in ONE shared warehouse across up to 4 floors. Robots navigate room-to-room, ride elevators to change floors, and detect items — all on one shared clock. Live HUD: robots, found/total, coverage %, elapsed; per-floor bot counts; play/pause/speed. Shows it's collecting jointly into the active project's environment (P3 data side already wired). Scales to tens of robots at interactive rate. - linked from cockpit VIEWS + browser nav. This is spec 04: launch fleets that collect data together from a space large enough — a warehouse with elevators.
…pecs Bump for the platform feature set (projects/environments, data standard, telemetry browser, backends incl. Isaac, fleet sim with warehouse+elevators, spatial perception+tracks, unified recall, project/env data isolation, standard env + camera registration). Built + twine-checked; ready to publish.
…ch-style)
The platform spine existed but the UX didn't reflect it. Now it's a real product:
- shell.js + ui.css: ONE persistent chrome on every dashboard page — left sidebar
(Home / Sims&Robots / Cockpit / Data{Browser,Scenarios,Timeline,Search,Analytics}
/ Fleet / Projects) + top bar with the PROJECT/ENVIRONMENT SWITCHER as the spine
(switch → everything re-scopes) + live/go-live badge. Auto-wraps each page's
content; pages dropped their bespoke navs. Retires project-chip.js.
- home.html at /: the Antioch-style console — active scope hero, environments grid
(open → launch), data KPIs, recent runs, quick actions. Clean go-live state on
the static host.
- setup.html at /setup: guided robot/sim setup — project → backend (live
availability) → robot+task or warehouse fleet → Launch (creates env, sets active,
routes to /sim?level= / /fleet-sim / connect). Replaces the tutorial-wall modal.
- Cockpit demoted to a workspace at /sim (/ is the dashboard now); its start modal
is a slim level-switcher with links back to Dashboard/Setup. build_site index =
home.html; vercel.json rewrites /sim → arena.html so the playable demo still
works. Data is divided by project/environment everywhere, not muddled.
277 passed, 1 skipped.
One agent per route, each scoped to only its page's files, reusing the shared ui.css tokens/components + the app shell — no shared-file edits, no collisions. - / (home): layered hero w/ live status, scannable env cards (backend/mode/runs/ last-activity), richer KPI tiles, status-dotted recent runs, strong offline state - /setup: real numbered wizard — gutter badges, ✓ selected states, backend caps, always-visible live summary + sticky Launch, responsive - /browser: dropped redundant scope banner, legible spatial map w/ legend + axes, scannable run rows, tidy backend matrix - /scenarios: bigger donuts, avg-duration KPI, tag facet, aligned dotted timelines, sticky-header runs table w/ Dur column - /search: controls grouped into a console card (prominent query + collapsible filters), polished result cards, dropped redundant :root overrides - /timeline: card-framed two-pane, outcome dots, clean selected state, responsive - /analytics: shared .kpi tiles, gradient bars + peak-marked sparkline, fleet grid - /projects: status-bar active scope, scannable project cards, contained camera tagging, strong empty state - /run: two-column synced player, grid/axis-labeled charts, 3-col event log; run-detail.js keeps renderRunRecord signature (timeline still works) - /fleet (swarm lab): rebuilt full-viewport overlay into a shell-friendly 3-column dashboard (ResizeObserver + screenToWorld fix), shared tokens - /fleet-sim: KPIs as .kpi tiles, legend, richer canvas (rooms/walls/labeled items/ elevators/robot trails, HiDPI) Verified: all 11 node --check clean; headless sweep = shell wraps every route, 11 nav items, zero orphaned content, zero console errors. Suite green.
The 11-route optimization moved nav into the centralized app shell and titles into the shell top bar, so the old assertions (per-page header text, per-page nav hrefs) were obsolete. Now: assert each dashboard loads shell.js + keeps the element ID its JS drives, and that shell.js carries the links to every route.
The /sim 'pick a robot & task' modal was hardcoded old-style hex (green-on-black, .22em letter-spacing) and clashed with the rest of the UI. Reskinned every modal class (start-card, bot-card, ros-card, fleet-card + controls) onto the ui.css tokens (--panel/--line/--fg/--fg-2/--fg-dim/--accent/--blue/--r) so the cockpit's launcher matches the dashboard. Also removed the local fake test projects (warehouse-pilot) so the project switcher reads 'scratch' again.
Was a long single-page scroll of all 4 steps. Now: a stepper (Project → Backend → What to run → Environment) showing ONE step at a time, with Back/Next and Launch only on the final step. Click a completed dot to jump back. Each step gets a short lead line so it's digestible. All IDs/logic preserved; verified headlessly — clean step transitions, zero errors.
…nify store path The user's point: dashboards should fill because you ran rapier, not from seeds. - /api/arena/state now indexes the sim's detections into the spatial store (throttled ~1 Hz, scoped to the active project/environment) — so just playing in rapier populates /search, /api/spatial (tracks) and /analytics with REAL, env-frame-positioned data. Previously detections only persisted if you started a recording and sealed it; the dashboards sat empty (which is why I'd seeded fakes). Verified: a sim detection is instantly searchable at its world pose. - Fixed a real bug: the searchable index used a CWD-relative '.roborun' while the recorder used '~/.roborun', so data split across two dirs. Unified the store to the same root as recorder.runs_root (honoring ROBORUN_STATE_DIR). - Wiped the 268MB of fake test observations + test runs → clean slate that only fills from real activity.
Replaces the 2D kinematic toy with an ACTUAL Rapier world (same engine as the cockpit): a warehouse of fixed-cuboid walls/rooms, N kinematic-capsule robots driven by a character controller so they collide with walls AND each other, navigating room to room. Top-down render of the true physics positions + trails. Every item a robot detects POSTs to the new /api/fleet/observe, which indexes it into the active project/environment store — so a fleet run fills /search and the spatial map with real, multi-robot, world-positioned data. Verified headlessly: Rapier WASM loads, robots physically move, coverage climbs, and the store grows from fleet detections. Old 2D fleet-sim.js removed.
Stacked floors in one Rapier world (6m apart in Y, so physically independent), each a ground slab + walls. N robots start on floor 0; ~25% of targets are the central elevator shaft — reaching it transitions a robot to the adjacent floor (brief ride, then teleport to the shaft on the destination floor). One top-down canvas per floor renders the true physics positions + per-floor bot counts. Detections across all floors POST to /api/fleet/observe into the active env. Verified headlessly: 3 floor canvases, robots ride the elevator (8/0/0 → 5/3/0 across floors), coverage climbs, store fills. floors/robots/speed controls live.
…ing + premature summary - /sim was a full-screen cockpit with no obvious way back (the shell can't wrap the immersive 3D view). Added a fixed top-left ← Dashboard button. - /setup: removed the redundant per-card step number (the top stepper already numbers steps — it read '1 ... 1') and stopped showing the full launch chain (dog · dog-sandbox on rapier → …) on earlier steps, which made it look like choices were already made. Earlier steps now show 'Step X of 4'; the full 'Ready to launch — …' summary appears only on the final step.
…apier physics) Keep them SEPARATE, not muddled. Doing rapier shows zero coordination-algorithm UI; the algorithm work lives in its own place. - Swarm Lab (/fleet) = layer 1: coordination algorithms — how a swarm decides where to spread & search, abstract, no physics. Reframed title + cross-link to Fleet Sim. - Fleet Sim (/fleet-sim) = layer 2: real Rapier physics — bodies, collisions, multi-floor + elevators, joint data. Scope line reframed + cross-link to Swarm Lab. No strategy/algorithm controls here. - Sidebar FLEET group reordered to read as the two layers (Swarm Lab → Fleet Sim). - Setup's fleet path still routes to the physics sim with no coordination UI.
Setup made you pick a robot+task, then the cockpit's detectMode() called showStart() unconditionally — reopening 'PICK A ROBOT & TASK' on top of the already-loaded level. Now showStart() is skipped when /sim?level= is present (you already chose); fresh /sim with no level still shows the picker. Verified end-to-end: Setup→Launch → /sim?level=dog-sandbox, no picker; bare /sim, picker.
The default was semantic (CLIP) search — but sim/fleet detections are stored label-only (no image → no embedding), so 'by meaning' returned 0 after a ~2s CLIP call (looked like it hung). Default to 'by object' (label): instant and it actually finds the data the platform generates. 'by meaning' stays available for camera/real data that has CLIP embeddings (tooltip notes the requirement).
…side/connect Picking Gazebo/Isaac/MuJoCo then launching dropped you into the in-browser Rapier cockpit as if it were that engine. Clarified the backend step: Rapier plays in the browser; MuJoCo/MJX/Gazebo/Isaac run server-side/externally (you connect to a running sim); real robot connects over ROS.
1) Don't auto-run the rapier fleet: it now starts PAUSED (▶ play); robots are placed but still until you press play. Fixed the elapsed timer to count sim time (only while playing), not wall-clock — so paused actually reads as paused. 2) Fleet (and its algorithm) is SEPARATE: removed 'Warehouse Fleet' from the single-robot 'What to run' picker in Setup — it doesn't belong next to Quadruped/Humanoid/Drone. Setup is single-robot/real only; a small link points to Fleet Sim for swarms. Cleaned the dead S.fleet code + launch branch. The coordination algorithm stays in the Swarm Lab (separate page), as before.
The fleet ran the real 3D Rapier world but only rendered flat 2D top-down
canvases — jarring next to the 3D cockpit. Render it in actual 3D instead:
- three.js scene of the stacked multi-floor warehouse: per-floor grids + slabs,
room walls as semi-transparent boxes, items as cylinders that light up green
on detection, the elevator drawn as a translucent shaft spanning floors.
- Robots are 3D bodies (capsule + heading nose) at true Rapier positions;
hidden while riding the elevator. Per-floor bot counts overlaid on the stage.
- Manual orbit camera (drag to rotate, scroll to zoom, gentle auto-rotate idle),
framed to centre the whole stack. Lights tuned so robots/walls read clearly.
- Easier to find: labelled '3D' in the Home hero actions and the sidebar
('Fleet Sim · 3D'), distinct from the Swarm Lab (the coordination algorithm).
window.__fleet exposes {S,V} for headless tests. Verified under headless WebGL:
131 static meshes + 8 robot meshes, robots move on play, items detect, 0 errors.
The 'white page' was never a blank/broken page — it was white leaking below
the content. Pages whose <style> forgot body{background} (analytics, timeline,
search, fleet) defaulted to a white body, and the shell's content column is
transparent, so any short page showed white under the panels. This is the same
'white page under timeline' reported earlier.
Fix it once, structurally: ui.css now sets html/body + .app-shell + .app-col to
var(--bg), so no page can leak white regardless of whether it sets its own body
background or how tall its content is. Verified: analytics/timeline/search/fleet
all render rgb(10,13,11) top-to-bottom.
New self-contained package `strata/` — one engine, two data models, on any S3-compatible endpoint (S3 / MinIO / R2 / B2 / local). Object storage is the source of truth; local RAM/NVMe is only cache. No external DB or lock service: serializable writes come from a CAS'd, numbered manifest log written straight to the bucket (PUT If-None-Match), the same design object-store table formats and serverless vector stores use. ReductStore feature parity (time-series blobs): buckets/entries/records (µs timestamp + labels + content-type + blob), write by timestamp + batched writes, read latest/at-ts, time-range + label queries, each_n/each_s downsampling, FIFO quota eviction, write-block sealing by size/records/age, bearer-token auth w/ per-bucket permissions, filtered + checkpointed replication, REST API. turbopuffer-class vectors + full-text: namespaces, upsert/delete with schemaless columnar attributes, exact NumPy ANN (cosine/euclidean/dot) over a per-namespace live view cached by manifest version, pushed-down attribute filters, BM25 full-text ranking, upsert shadowing (latest wins) via segment-seq. Layout: backend (Local/S3/Memory + put_if_absent CAS), manifest (commit log + checkpoints + optimistic retry), segment (immutable blob + columnar vector), blobstore, vectors, query (shared filter AST), retention, auth, replication, server (REST + idle-block ticker), client, cli (`strata serve`). Spec + parity matrices in docs/STRATA_SPEC.md. Tests (tests/test_strata.py, 26): both backends + CAS, segment roundtrips, manifest commit/checkpoint/concurrent-writers (4 threads, 80 commits, 0 lost), blob write/read/query/downsample/FIFO/durability, vector ANN/metrics/shadow/ delete/BM25/dim-check, auth, filter language, server+client e2e, auth enforcement (401/403), replication (filtered+idempotent), and the WHOLE engine running on the S3 backend via a fake S3 client (incl. If-None-Match). Full repo suite: 286 passed. Packaging: `strata` console script + [strata] extra (numpy, boto3).
- Compaction: blobs merge small segments into size-bounded blocks; vectors merge to one segment and physically drop shadowed + tombstoned rows. Both are conflict-free with concurrent writers (blobs key on time; the merged vector segment inherits the max sequence it replaces so newer writes still shadow it). Exposed as POST .../compact on blobs and vectors + client methods. - Hybrid search: passing both `vector` and `rank_by` fuses vector ANN and BM25 via reciprocal-rank fusion (turbopuffer-style). - Fix: MemoryObjectStore.put_if_absent now locks so it's an atomic CAS — without it concurrent commits could both pass the existence check and lose a write (surfaced under full-suite load; real S3 If-None-Match is atomic server-side). Concurrent-writer test stress-passed 10×. Strata suite: 31 tests; full repo: 289 passed.
Adds a cached IVF index (k≈√n k-means clusters) for approximate nearest-neighbour search at scale, alongside the exact default: - query(..., approx=True, nprobe=N): scores only the nprobe nearest clusters. Measured 7.6x faster than exact at 200k vectors; recall is tuned by nprobe (higher → recall → 1.0) — the same recall/latency dial turbopuffer exposes. - Exact stays the default (ground truth, sub-ms to ~50k). When there's no filter the approx path uses the IVF candidate set directly (skips a costly intersect). - IVF index cached per manifest version, rebuilt only when the namespace changes. - approx/nprobe plumbed through the REST query endpoint + Python client. Tests: recall >=0.7 at default-ish nprobe and >=9/10 at high nprobe (near-exact). Strata suite 30; full repo 290 passed.
… /studio A React/Vite SPA in app/ (builds to roborun/web/studio/) that unifies the previously-siloed pages behind one shell with source-agnostic panels (LiveSource / RunSource), a global scrubber, and project scope. - Front door: server.py redirects / and the old standalone routes into /studio. - Live: first-run welcome + CTA, filtered events, framed empty states. - Sims: thin launcher (Arena/Fleet/Data/Real); in-app behavior editor (hot-reload); Real robot = native rosbridge connect; embedded pages strip standalone nav. - Runs: human-readable list + provenance bar with live verify + reversible tamper demo. - Search: thumbnails, replayable affordance, honest CLIP gating (per-dim search, real embeddings only); drop-zone to import a .mcap and replay it. - Agent (MCP connect + live activity), native Scenarios + Analytics, guided tour. Backend fixes: sim/state _drone_ctrl crash; arena no longer silently auto-records; run_series/frame_at tolerate corrupt/partial MCAPs; arena live obs carry run_id; /api/run/upload, /api/demo/seed, /api/search/caps. Full audit in docs/STUDIO_AUDIT.md.
…ow, accent-fill) Imported the official RoboRun Design System (claude.ai/design f84aeaa8) via the claude_design MCP and re-skinned Studio to its visual language: - dashboards set in the system monospace stack (dropped IBM Plex webfonts) - flat layered surfaces + a single soft shadow — removed all glow - --accent-fill active states, DS radii (10/6/999), 232px sidebar, 1280px cap - welcome is the one tinted hero (radial accent wash); rr-pulse live dot Kept the consolidated nav. See docs/STUDIO_AUDIT.md Pass 7.
… flow The scope chip was hidden for newcomers, so there was no way to create or switch a project. Now it's always shown (top bar): "+ New project" creates a project + its default environment and switches to it; existing projects/envs are listed to switch between; "use scratch" clears scope. Explains scratch plainly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A multi-quadruped /fleet sandbox reachable from the robot picker — built to teach how a swarm covers ground over an imperfect radio, not just demo it.
Fleet sandbox (
/fleet)POST /api/fleet/strategy), runnable on the spot.Python twin
roborun/swarm/— the same comms model, strategies, base relay and data points as runnable Python (python -m roborun.swarm); ships with the package.Also
confirm()(deploy-to-robot) with the styled in-app modal.vercel.json(cleanUrls) so/fleetresolves on the static Vercel build.Verification
/api/fleet/strategy,/api/sources/scan) all checked.Note: a pre-existing uncommitted Vercel-analytics change in
roborun/web/runtime-base.jswas intentionally left out of this PR.🤖 Generated with Claude Code