Dedupe the lexer numeric-suffix list#75
Merged
StreamDemon merged 1 commit intoJul 2, 2026
Merged
Conversation
`numeric_suffix` and `validate_numeric_body` each carried their own copy of the 13-entry suffix table, so any future suffix change had to be made twice or the two paths would drift apart. Hoist the table into a shared `NUMERIC_SUFFIXES` const and return the matched `&'static str` instead of allocating a fresh `String` on every suffix scan. A new test iterates the shared table and lexes `1<suffix>` for every entry, so additions to the list are covered automatically.
There was a problem hiding this comment.
No issues found across 1 file
Confidence score: 5/5
- Automated review surfaced no issues in the provided summaries.
- No files require special attention.
Architecture diagram
sequenceDiagram
participant Lexer as sploosh-lexer Lexer
participant Suffixes as NUMERIC_SUFFIXES const
participant Token as Token
Note over Lexer,Token: Numeric literal lexing (unchanged flow, source of truth deduped)
Lexer->>Lexer: scan_numeric_literal()
Lexer->>Suffixes: CHANGED: iterate &NUMERIC_SUFFIXES
Suffixes-->>Lexer: &'static str suffix match (was String allocation)
alt Suffix found
Lexer->>Lexer: advance pos by suffix.len()
Lexer-->>Token: produce IntLit / FloatLit
else No suffix
Lexer-->>Token: produce numeric literal without suffix
end
Note over Lexer,Suffixes: Separator validation also uses same const
Lexer->>Lexer: validate_numeric_body(start, base)
Lexer->>Suffixes: NEW: strip suffix via &NUMERIC_SUFFIXES
Suffixes-->>Lexer: body without suffix
Note over Lexer,Token: Table-driven test covers all suffixes
participant Test as every_numeric_suffix_lexes test
Test->>Suffixes: iterate NUMERIC_SUFFIXES
loop for each suffix
Test->>Lexer: lex("1{suffix}")
Lexer-->>Test: tokens[0] (kind, lexeme, span)
Test->>Test: assert kind (IntLit / FloatLit)
Test->>Test: assert lexeme == "1{suffix}"
Test->>Test: assert span correct
end
Auto-approved: Refactors numeric suffix list into shared constant and returns &'static str instead of allocating, with new table-driven tests. No behavior change, low risk.
Re-trigger cubic
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
numeric_suffixandvalidate_numeric_body— into a single sharedNUMERIC_SUFFIXESconst insploosh-lexer.numeric_suffixnow returns the matched&'static strinstead of allocating aStringper scan.1<suffix>for every entry in the shared list (kind + lexeme + span asserted), so future suffix additions are covered automatically.First sub-PR of the parser enhancement wave (targets
feature/parser-enhancements-base, see PR #71 roadmap: "Dedupe the numeric-suffix list in the lexer"). No behavior change.Related Issue
None (PR #71 roadmap item).
Spec Sections Affected
None — implementation quality only.
Checklist
docs/pages — N/A, no behavior changeBuild Targets Tested
cargo fmt --all -- --check,cargo clippy --workspace --all-targets -- -D warnings,cargo test --workspaceall green locally (49 tests).Test Plan
every_numeric_suffix_lexestest iteratesNUMERIC_SUFFIXESand asserts token kind (IntLit/FloatLit), lexeme, and span for each suffix.lexes_suffixed_numbers,rejects_int_suffix_on_float_literals,rejects_float_suffix_on_based_literals,separator_validation_is_base_aware) continue to pass unchanged.Summary by cubic
Deduplicates the lexer numeric-suffix list by moving it into a shared
NUMERIC_SUFFIXESconst used by both suffix scanning and separator validation, and returns&'static strto avoid per-scan allocation. Adds a table-driven test that lexes1<suffix>for every entry; behavior is unchanged.Written for commit e31a214. Summary will update on new commits.