Skip to content

repl: add inline syntax highlighting#64609

Open
hemanth wants to merge 1 commit into
nodejs:mainfrom
hemanth:repl/syntax-highlighting
Open

repl: add inline syntax highlighting#64609
hemanth wants to merge 1 commit into
nodejs:mainfrom
hemanth:repl/syntax-highlighting

Conversation

@hemanth

@hemanth hemanth commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Split from #64443 per review feedback to use idiomatic single-feature PRs.

Uses the Acorn tokenizer (already in deps/) to colorize JavaScript input in real-time:

  • Keywords β†’ Magenta
  • Strings β†’ Green
  • Numbers β†’ Yellow
  • Regular expressions β†’ Red
  • Comments β†’ Gray

Highlighting is display-only β€” repl.line stays plain text. Can be disabled via syntaxHighlighting: false or TERM=dumb.

Dogfoods util.styleText per @edsadr's suggestion.

Refs: #48164

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. repl Issues and PRs related to the REPL subsystem. labels Jul 19, 2026

@avivkeller avivkeller left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few concerns here.

  1. Node provides util.inspect.styles which defines colors, so we don't need to define them separately
  2. This involves hooking into and monkey patching REPL internals. Rather than monkey patching them, they should be adapted to work with new code.
  3. This is a lot of code to maintain, and it'll require manual editing every time a new keyword is added to V8. AST parsing provides some of that info for us, and we can probably reduce overhead by relying on it.

(Note that I have an alternate implementation at #64591. That PR and this one being opened around the same time is a coincidence of the REPL inspector landing)

@hemanth
hemanth force-pushed the repl/syntax-highlighting branch from b39fe8f to 756e16e Compare July 19, 2026 23:07
@hemanth

hemanth commented Jul 19, 2026

Copy link
Copy Markdown
Contributor Author
  • Colors now come from util.inspect.styles via stylizeWithColor β€” no more hardcoded color names
  • Dropped the _writeToOutput/_refreshLine monkey-patch, using readline's colorize hook instead
  • Tokenβ†’style mapping uses Acorn's token.type.keyword/label β€” no manual keyword list to maintain

For context: this is a split from #64443 which predates #64591. Will let you all decide which PR to pick 🀝

PTAL @avivkeller

Comment thread lib/internal/repl/utils.js Outdated
Comment on lines +146 to +236
const { stylizeWithColor } = require('internal/util/inspect');
const { Parser } = require('internal/deps/acorn/acorn/dist/acorn');

const tokenizer = FunctionPrototypeBind(Parser.tokenizer, Parser);

/**
* Maps an Acorn token to a util.inspect style name.
* Returns undefined for tokens that should not be highlighted.
* @param {object} token The Acorn token.
* @returns {string|undefined} The inspect style name.
*/
function tokenStyle(token) {
const { label, keyword } = token.type;

if (keyword !== undefined) {
if (label === 'true' || label === 'false') return 'boolean';
if (label === 'null') return 'null';
if (label === 'import' || label === 'export') return 'module';
return 'special';
}

switch (label) {
case 'name':
switch (token.value) {
case 'undefined':
return 'undefined';
case 'NaN':
case 'Infinity':
return 'number';
default:
return undefined;
}
case 'num':
return 'number';
case 'bigint':
return 'bigint';
case 'string':
case 'template':
case '`':
return 'string';
case 'regexp':
return 'regexp';
default:
return undefined;
}
}

/**
* Applies ANSI syntax highlighting to a JavaScript code string.
* Uses the Acorn tokenizer and util.inspect.styles for colors,
* so the color scheme stays in sync with util.inspect output.
* @param {string} code The JavaScript code string to highlight.
* @returns {string} The highlighted code string.
*/
function syntaxHighlight(code) {
if (code.length === 0) return code;

let result = '';
let offset = 0;

function write(start, end, inspectStyle) {
result +=
StringPrototypeSlice(code, offset, start) +
stylizeWithColor(StringPrototypeSlice(code, start, end), inspectStyle);
offset = end;
}

try {
const iterator = tokenizer(code, {
__proto__: null,
allowHashBang: true,
ecmaVersion: 'latest',
onComment(_block, _text, start, end) {
write(start, end, 'undefined');
},
});

for (const token of iterator) {
const inspectStyle = tokenStyle(token);
if (inspectStyle !== undefined) {
write(token.start, token.end, inspectStyle);
}
}
} catch {
// Acorn throws for unfinished strings, templates, and comments.
// Tokens reported before the error are still useful while typing.
}

return result + StringPrototypeSlice(code, offset);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I respect that you changed this following my review, this code is plagiarized from my own. It's important to see your own spin on things.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @avivkeller β€” "plagiarized" is a strong word. The original implementation was in #64443 which you reviewed and suggested splitting into separate PRs. While I was working on that split, you opened #64591 independently. When your review on this PR suggested using stylizeWithColor and the colorize hook, I adopted those patterns from your PR to address your feedback β€” that felt like the right thing to do.

If the code ended up too close to yours, a co-author credit would have been the collaborative path forward. That said, you are one of the maintainers β€” feel free to close this PR if you'd prefer to go with #64591.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think closing this is the right call. My PR doesn't get any special priority just because I'm a maintainer, as we treat all PRs equally, and that includes mine.

That said, regarding your statements:

  1. "I adopted those patterns from your PR." Adopting patterns means using existing code as a reference to write something new. What actually happened here is that my changes were inserted into yours almost 1:1 (which is also why the lint is failing).

  2. "if the code ended up too close to yours": if you'd referenced my code and it came out similar, that would be completely fine. Referencing each other's work is a normal part of collaboration. Copying it verbatim is a different thing.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally understand, it was a cherry-pick there, as I was addressing your review wanted to use the same patterns you have defined.

What do you suggest we do with PR?

  • Do a new implementation of the same challenge?
  • Close this on behalf?
  • Revert this PR to the pervious state and enhance?
  • Any other alternatives.

@hemanth
hemanth force-pushed the repl/syntax-highlighting branch 2 times, most recently from 53b4780 to 405267e Compare July 19, 2026 23:41
Use the Acorn tokenizer to colorize JavaScript input in real-time.
Keywords are magenta, strings green, numbers yellow, regexps red,
and comments gray. Highlighting is display-only β€” repl.line stays
plain text.

Can be disabled via `syntaxHighlighting: false` in repl.start() or
TERM=dumb.

Refs: nodejs#48164
Signed-off-by: hemanth <hemanth.hm@gmail.com>
@hemanth
hemanth force-pushed the repl/syntax-highlighting branch from 405267e to f74ff7b Compare July 21, 2026 06:18
@codecov

codecov Bot commented Jul 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 68.36735% with 31 lines in your changes missing coverage. Please review.
βœ… Project coverage is 90.10%. Comparing base (0bcc6ef) to head (f74ff7b).
⚠️ Report is 15 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/repl/utils.js 67.39% 30 Missing ⚠️
lib/repl.js 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #64609      +/-   ##
==========================================
- Coverage   90.14%   90.10%   -0.04%     
==========================================
  Files         741      741              
  Lines      242080   242189     +109     
  Branches    45563    45572       +9     
==========================================
+ Hits       218213   218226      +13     
- Misses      15375    15472      +97     
+ Partials     8492     8491       -1     
Files with missing lines Coverage Ξ”
lib/repl.js 92.87% <83.33%> (-0.05%) ⬇️
lib/internal/repl/utils.js 94.12% <67.39%> (-2.80%) ⬇️

... and 39 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. repl Issues and PRs related to the REPL subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants