Scenario yaml source-of-truth + on_simulation_end verdict override

[theomonnom]

No description provided.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
14 out of 19 committers have signed the CLA.

✅ charlotte-zhuang
✅ flynn-hamming
✅ he-yufeng
✅ bcherry
✅ toubatbrian
✅ u9g
✅ longcw
✅ kamil-bidus
✅ Topherhindman
✅ russellmartin-livekit
✅ adrian-cowham
✅ sumanyu
✅ harshitajain165
✅ mitrushchienkova
❌ sgu-bithuman
❌ FLoppix
❌ brycedli
❌ nightcityblade
❌ renal128
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[github-actions]

STT Test Results

⚠ Could not parse test results: [Errno 2] No such file or directory: 'test-results.xml'

---

Triggered by workflow run #2299

[devin-ai-integration]

Devin Review found 2 potential issues.

View 9 additional findings in Devin Review.

[devin-ai-integration]

🔴 Soniox STT enters infinite reconnect loop after normal stream completion

The new reconnect-on-WS-drop logic at the end of _recv_messages_task unconditionally sets self._reconnect_event whenever the async for msg in self._ws loop terminates — including after a normal finished message from the server. This causes the _run() method's while True loop to reconnect instead of breaking. On the second connection, self._input_ch is already closed (user called end_input()), so send_task immediately exhausts, sends close, and recv_task finishes, setting the reconnect event again — creating an infinite reconnect loop that prevents _run() from ever returning.

How the loop forms

1. Normal completion: server sends finished, closes WS
2. async for msg in self._ws ends → falls through to the new self._reconnect_event.set()
3. In _run(), both send_recv_group and wait_reconnect are done
4. wait_reconnect in done → loop doesn't break, clears event, reconnects
5. New send_task sees closed _input_ch → immediately finishes → sends close
6. New recv_task sees close ack → finishes → sets reconnect event again
7. Back to step 3 — infinite loop

The stream's _main_task hangs until aclose() cancels it, leaking the connection and preventing the base class retry loop from completing.

Prompt for agents

The reconnect-on-drop logic in _recv_messages_task fires unconditionally when the async-for loop exits, including after a normal server-sent `finished` message. This creates an infinite reconnect loop because _input_ch is already closed on the second connection.

The fix should guard the reconnect request so it only fires on unexpected WebSocket drops (not after a clean `finished` completion). One approach: track whether a `finished` message was received (e.g. set a `self._finished = True` flag inside the `if content.get('finished')` block), and only set the reconnect event when the WS closes WITHOUT having received `finished`. Alternatively, return early from _recv_messages_task after processing `finished` to avoid reaching the reconnect code.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🚩 Cartesia STT default model changed from ink-whisper to ink-2 with turn detection

The Cartesia STT plugin now defaults to ink-2 (for English) instead of ink-whisper. ink-2 supports server-side turn detection (AutoFinalizeRecognizeStream using the /stt/turns/websocket endpoint), interim results, and preflight transcripts — but NOT word-level aligned transcripts. ink-whisper is preserved as a fallback for non-English languages and remains available via the legacy LegacyRecognizeStream. The capabilities are now set dynamically based on the resolved model (interim_results=True for ink-2, aligned_transcript='word' for ink-whisper). Users who relied on cartesia.STT() producing word timestamps will silently lose them unless they explicitly pass model='ink-whisper'.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

Devin Review found 2 new potential issues.

View 12 additional findings in Devin Review.

[devin-ai-integration]

🚩 Speechmatics default turn_detection_mode changed from ADAPTIVE to EXTERNAL requires silero

The default turn_detection_mode changed from TurnDetectionMode.ADAPTIVE to TurnDetectionMode.EXTERNAL at livekit-plugins-speechmatics/stt.py:88. In EXTERNAL mode, the constructor auto-loads Silero VAD (line 278-286), which requires livekit-plugins-silero to be installed. This package is not a declared dependency of the speechmatics plugin (pyproject.toml only depends on livekit-agents>=1.5.18 and speechmatics-voice[smart]>=0.2.8). Users upgrading who relied on the old ADAPTIVE default will get an ImportError at runtime if silero is not installed. The error message is descriptive and tells users how to fix it (install silero, or pass vad=None, or use turn_detection_mode=ADAPTIVE). This pattern is consistent with inference.STT for Speechmatics models. Still, it's a breaking change in default behavior that could surprise users on upgrade.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🚩 wait_for_idle / _wait_for_idle_and_hold idle-hold mechanism may not block other callers

The _wait_for_idle_and_hold context manager at agent_session.py:1376-1391 increments _idle_holds, clears _idle_released, and sets _IdleHoldContextVar. However, the public wait_for_idle method at line 1356-1374 delegates to activity.wait_for_idle() and never checks _idle_holds or waits on _idle_released. This means the "hold" may only work if AgentActivity.wait_for_idle() checks _IdleHoldContextVar (which I couldn't fully verify in this review). If the AgentActivity side doesn't participate, other callers of session.wait_for_idle() would not be blocked during the hold window, which could cause tool reply coalescing races. The _idle_released event and _idle_holds counter would be dead code in that case.

---

Was this helpful? React with 👍 or 👎 to provide feedback.