Add UTF-8-safe base64 encoding for code blocks#2381
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds UTF-8-safe Base64 helpers to the engine’s V8 isolate bootstrap so code blocks can Base64-encode/decode arbitrary Unicode strings (not just Latin1), and ensures code blocks type-check cleanly by emitting matching ambient type declarations during compilation.
Changes:
- Extend the isolate base64 polyfill with
btoaUtf8/atobUtf8(guarded, additive). - Emit
openops-globals.d.tsinto each code-block compile directory sotscrecognizes the new globals. - Add isolate-level tests validating UTF-8 encode/decode and round-trips for Unicode data.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| packages/engine/test/core/code/v8-isolate-code-sandbox.test.ts | Adds coverage for UTF-8 base64 helpers in the isolate sandbox. |
| packages/engine/src/lib/core/code/polyfills/base64-polyfill.ts | Introduces btoaUtf8/atobUtf8 runtime polyfills and keeps their ambient type declarations alongside the polyfill. |
| packages/engine/src/lib/code-block/code-builder.ts | Writes openops-globals.d.ts during code-block compilation so tsc accepts the new globals. |
ravikiranvm
approved these changes
Jun 26, 2026
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Part of OPS-4569.
Additional Notes
Code blocks running in the V8 isolate get a pure-JS base64 polyfill (
btoa/atob). Those built-ins are Latin1-only and throwthe string to be encoded contains characters outside of the Latin1 rangewhenever the input contains non-Latin1 characters (accented text, currency symbols, CJK, emoji), so any block that base64-encodes user data fails on such input.This PR:
btoaUtf8/atobUtf8helpers to the base64 polyfill that is bootstrapped into every code-block isolate. They are additive and behindtypeofguards — existingbtoa/atobbehavior is unchanged.tscbefore they run in the isolate, so without a declarationtscrejects any block referencing the helpers withTS2304: Cannot find name 'btoaUtf8'. The declarations live next to the runtime polyfill (BASE64_POLYFILL_TYPES) and are written asopenops-globals.d.tsinto each block's compile directory by the code builder, keeping runtime and types in sync.Testing Checklist
Check all that apply:
I tested the feature thoroughly, including edge cases
I verified all affected areas still work as expected
Automated tests were added/updated if necessary
Changes are backwards compatible with any existing data, otherwise a migration script is provided
Testing notes
v8-isolate-code-sandbox.test.tscoveringbtoaUtf8/atobUtf8encode, decode, a unicode round-trip (Coût € — 日本語 🚀), and aJSON.parse(atobUtf8(...))decode of an encoded payload.tsc --build: a block callingbtoaUtf8compiles withopenops-globals.d.tspresent and fails withTS2304without it.npx nx lint engineandnpx nx test enginepass.