Skip to content

Make Token and TokenKind Copy, drop parse-loop clones#77

Merged
StreamDemon merged 1 commit into
feature/parser-enhancements-basefrom
feature/parser-enh-copy-tokens
Jul 2, 2026
Merged

Make Token and TokenKind Copy, drop parse-loop clones#77
StreamDemon merged 1 commit into
feature/parser-enhancements-basefrom
feature/parser-enh-copy-tokens

Conversation

@StreamDemon

@StreamDemon StreamDemon commented Jul 2, 2026

Copy link
Copy Markdown
Owner

Summary

  • Derives Copy on Token and TokenKind. After Store spans instead of lexeme strings on tokens #76 a token is a payload-free kind (the only payload, Keyword, is already Copy) plus a Span — both trivially copyable.
  • Removes every .clone() from the parse loops: at/eat/expect, bump, peek_kind/peek_kind_at, args, the balanced-skip helpers, and recover_until now all pass tokens and kinds by value.
  • can_begin_expr takes TokenKind by value, so the send-head check passes it point-free.

Third sub-PR of the enhancement wave. Re-slice note: the PR #71 roadmap sketched this slot as "at/eat/expect take &TokenKind". Deriving Copy reaches the same goal — no clones in any parse loop — with by-value call sites instead of reference threading; it only became possible once #76 removed the owned lexeme. No behavior change.

Related Issue

None (PR #71 roadmap item, re-sliced as described above).

Spec Sections Affected

None — implementation quality only.

Checklist

  • Code follows the Sploosh design principles (one way to do it, explicit over implicit, etc.)
  • Documentation updated in relevant docs/ pages — N/A, no behavior change
  • Tests added or updated — N/A, pure representation change; suite unchanged and green
  • All build targets still compile (if applicable)
  • Spec-only PR (skip Build Targets section if checked)

Build Targets Tested

  • cargo fmt --all -- --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace all green locally (49 tests).

Test Plan

  • Entire existing suite (11 lexer + 37 parser + corpus) passes unchanged — the change is invisible to parse results by construction, and clippy's clone_on_copy lint (denied in CI) guarantees no stale clone survives.

Summary by cubic

Make Token and TokenKind Copy and remove clone calls in parser loops by passing tokens and kinds by value. This simplifies the parser and avoids unnecessary allocations with no behavior change.

  • Refactors
    • Derived Copy on Token and TokenKind (token = kind + Span, both copyable).
    • Removed .clone() across parse loops: at/eat/expect, bump, peek_kind/peek_kind_at, args, balanced-skip helpers, and recover_until.
    • can_begin_expr now takes TokenKind by value; simplified the send head check.

Written for commit 45dc99c. Summary will update on new commits.

Review in cubic

With the lexeme gone, a token is a payload-free kind plus a span — both
trivially copyable. Deriving Copy lets every parse loop pass and return
tokens by value, so the `.clone()` calls sprinkled through `at`, `eat`,
`expect`, `bump`, `peek_kind`, and the recovery helpers all disappear.

The PR #71 roadmap sketched this slot as "`at`/`eat`/`expect` take
`&TokenKind`"; deriving Copy reaches the same goal (no clones in parse
loops) with by-value call sites instead of reference threading, which
only became possible after the span-slicing change landed.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Auto-approved: Refactors token representation to be Copy, removing clone calls in parser loops. No behavior change, all tests pass.

Re-trigger cubic

@StreamDemon StreamDemon merged commit 018bf45 into feature/parser-enhancements-base Jul 2, 2026
2 checks passed
@StreamDemon StreamDemon deleted the feature/parser-enh-copy-tokens branch July 2, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant