feat(tts): add Microsoft Edge TTS provider#168
Conversation
|
@hagope thanks for the PR! The overall integration looks clean, and the config/provider wiring makes sense. Build, lint, and tests pass after installing the new dependency.
However, the upstream edge-tts implementation opens a separate WebSocket synthesis stream per chunk. I also smoke-tested this against speech.platform.bing.com: short text works, but longer text that gets split into multiple chunks times out after 60s. This can affect real bot usage because the bot limits TTS input by characters (MAX_TTS_INPUT_CHARS = 4000), while Edge splits by UTF-8 bytes (4096). Non-ASCII text can easily exceed 4096 bytes while still being under 4000 characters. Could you please adjust the implementation to synthesize each chunk using a separate WebSocket connection, then concatenate the resulting MP3 buffers? Alternatively, at minimum, the code should not treat close as success unless all chunks have completed. |
Adds a fourth TTS provider that uses Microsoft Edge's online Read Aloud service via WebSocket. No API key, account, or TTS_API_URL required; only outbound HTTPS/WebSocket access to speech.platform.bing.com. Implementation ports the protocol from the Python edge-tts reference (https://github.com/rany2/edge-tts): - Sec-MS-GEC DRM token: SHA256 of Windows file-time ticks (rounded to 5 min) + trusted client token, with single 403 retry that corrects clock skew from the server Date header - SSML framing with prosody rate/volume/pitch and 4096-byte UTF-8-safe chunking that never splits multi-byte chars or XML entities - WebSocket message handling: binary frames use a 2-byte big-endian header-length prefix (length includes the trailing CRLF), audio starts at offset 2 + headerLength; text frames signal turn.end per chunk Adds ws as a runtime dependency (Node 20+ target; global WebSocket is only stable from Node 22+). - New: src/app/services/edge-tts.ts + tests - Wire 'edge' provider into config.ts and tts-service.ts - isTtsConfigured() returns true for edge (no credentials needed) - Document provider in .env.example and PRODUCT.md
The service does not reliably accept a second SSML turn on the same socket: multi-chunk input (possible because the bot's 4000-char limit can exceed Edge's 4096-byte chunk limit for non-ASCII text) hung until the 60s timeout. Open a fresh connection per chunk, as the upstream edge-tts client does, and concatenate the resulting MP3 buffers. A close before turn.end now always rejects instead of returning partial audio as success. Also stub TTS_PROVIDER/TTS_MODEL in the config default-values test so ambient environment variables no longer leak in. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Keep an error sink on the WebSocket during teardown: in ws 8.x, close() on a still-connecting socket emits 'error' on the next tick, and with listeners removed that was an uncaught exception that crashed the process on every 403 or pre-connect timeout. - Bound the whole synthesis (all chunks) by one shared deadline wired from tts-service's TTS_REQUEST_TIMEOUT_MS, matching the other providers instead of allowing 60s per chunk. - Fix binary-frame bounds check: header occupies [2, 2+headerLength), so guard against 2 + headerLength > buf.length. - Escape voice/rate/volume/pitch when interpolated into SSML attributes so a TTS_VOICE with ' or & fails loudly, not as malformed XML. - Drop the unreachable edge branch in getNotConfiguredMessage (isTtsConfigured is always true for edge) and the dead throw after the 403 retry; write the clock-skew update as an absolute assignment; reuse EDGE_DEFAULT_VOICE in tts-service. - Simplify removeIncompatibleCharacters to a regex replace, jsDateString to a toUTCString reorder, and the UTF-8 split probe to a continuation-byte check. - Add tests: SSML attribute escaping, fresh-connection 403 retry, and the shared cross-chunk deadline. Verified end-to-end against the live service: 3-chunk (8.9KB) input synthesizes a valid MP3 in ~6s across 5 consecutive runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
430ba7e to
1332453
Compare
|
Thanks for the thorough review and the smoke test — you were right, the shared-socket design was the problem. Fixed and rebased onto Multi-chunk: each chunk now gets its own WebSocket connection, matching upstream edge-tts, and the MP3 buffers are concatenated in order. The partial-audio edge case is fixed too: a close before A few more hardening fixes that came out of re-reviewing the module:
Verified end-to-end against the live service: an 8.9 KB input → 3 chunks → valid concatenated MP3 in ~6s, consistent across 5 consecutive runs (previously this exact case hit the 60s timeout). New tests cover per-chunk connections, ordered concatenation, partial-audio rejection, the fresh-connection 403 retry, and the shared deadline. |
|
@hagope thanks for contribution! |
Summary
Adds Microsoft Edge TTS as a fourth TTS provider (
TTS_PROVIDER=edge). It uses Microsoft Edge's online Read Aloud service via WebSocket — no API key, account, orTTS_API_URLrequired; only outbound HTTPS/WebSocket access tospeech.platform.bing.com.Implementation ports the protocol from the Python edge-tts reference.
How it works
Sec-MS-GEC): SHA256 of Windows file-time ticks (rounded to 5 min) + trusted client token. Includes a single 403 retry that corrects clock skew from the server'sDateheader.turn.endalways rejects; partial audio is never returned as success.TTS_REQUEST_TIMEOUT_MS) bounds the whole synthesis across all chunks, matching the other providers' timeout semantics.\r\n), so audio starts at offset2 + headerLength.audio-24khz-48kbitrate-mono-mp3.Changes
src/app/services/edge-tts.ts+tests/app/services/edge-tts.test.ts'edge'provider intosrc/config.tsandsrc/app/services/tts-service.tsisTtsConfigured()returnstruefor edge (no credentials needed).env.exampleandPRODUCT.mdwsas a runtime dependency (Node 20+ target; the globalWebSocketis only stable from Node 22+)Configuration
Verification
npm run build/npm run lint/npm testall pass (1088 tests)