Skip to content

Add 100% test coverage floor with SimpleCov#2072

Merged
zkoppert merged 9 commits into
masterfrom
test-coverage-100
Jun 8, 2026
Merged

Add 100% test coverage floor with SimpleCov#2072
zkoppert merged 9 commits into
masterfrom
test-coverage-100

Conversation

@zkoppert
Copy link
Copy Markdown
Member

@zkoppert zkoppert commented Jun 8, 2026

Why

Test coverage on github/markup was at 90.23% line / 64.71% branch with
no enforcement in CI. Every implementation file had at least one untested
branch, and there was nothing to stop a future change from quietly dropping
coverage further. Before tackling larger refactors (the in-flight
commonmarker 2.x migration in #2059, in particular), I wanted a coverage
floor we can trust.

What changed

  • Added SimpleCov as a development dependency (~> 0.22) and wired it
    up in a new test/test_helper.rb with enable_coverage :branch,
    track_files "lib/**/*.rb", and minimum_coverage line: 100, branch: 100.
    No new CI step needed: SimpleCov's at_exit hook calls Kernel.exit with
    a non-zero status when the floor is breached, which propagates through
    the existing bundle exec rake step.
  • Wrote 28 targeted tests in test/coverage_test.rb covering every
    previously-untested branch: the render-fallback path, the render_s
    guard, the markup_impl duplicate guard, every Markdown loader proc
    (BlueCloth / Kramdown / RDiscount / Redcarpet / Maruku /
    GitHub::Markdown), every CommandImplementation block arity, the
    non-zero-exit error path, the empty-output fallback, the language /
    match? paths when Linguist is absent, the Implementation constructor
    when Linguist is absent plus its invalid-language raise, and the
    WikiCloth ESCAPED_TAGS << 'tt' idempotency guard.
  • Refactored lib/github/markups.rb from a bare DSL (loaded via
    instance_eval File.read(...)) to explicit GitHub::Markup.markup(...) /
    command(...) / markup_impl(...) calls so that require_relative can
    load the file. The old form was invisible to Ruby's Coverage module
    (and therefore to SimpleCov). Registration order and @@markups
    contents are unchanged - this is a pure mechanism swap.
  • Read VERSION from lib/github-markup.rb source via regex in the gemspec
    instead of require-ing the library. Requiring it at gemspec evaluation
    time (which runs before SimpleCov starts via bundler/gem_tasks) hid
    every line of the file from coverage. The regex now also fails loudly
    with the file path if it ever stops matching.
  • Marked the legacy RDoc::Markup::ToHtml.new zero-arg fallback in
    lib/github/markup/rdoc.rb with :nocov: markers. That branch only
    fires on RDoc < 4, and Ruby 3.3 / 3.4 / 4.0 all ship RDoc 6.x in the
    stdlib, so it's unreachable on every CI matrix entry.
  • Added coverage/ to .gitignore.

Each change is a separate signed commit so the history reads cleanly:

SHA Purpose
6900a6f Explicit module references in lib/github/markups.rb
149bafd Read VERSION from source instead of require-ing the lib
6f469d1 Exclude legacy RDoc < 4 fallback from coverage
f8f7df0 Add SimpleCov with a 100% line + branch floor
05d38d8 Ignore SimpleCov coverage/ output
2a07090 Fail loudly if VERSION regex doesn't match
5a3eb25 Strengthen MediaWiki idempotency assertion

Testing

bundle exec rake test on Ruby 3.4.6 locally:

49 runs, 110 assertions, 0 failures, 0 errors, 0 skips
Line Coverage:   100.0% (202 / 202)
Branch Coverage: 100.0% (37 / 37)

Before/after:

Metric Before After
Line coverage 90.23% 100.0%
Branch coverage 64.71% 100.0%
Tests 21 49
Assertions ~25 110
CI enforcement none minimum_coverage line: 100, branch: 100

I also ran a three-model code review (Claude Opus 4.7, Claude Sonnet 4.6,
GPT 5.4) over the full diff before opening this PR; the synthesis is in
.git/copilot-pr-review/test-coverage-100.md locally. All three returned
no significant issues. Two cheap robustness improvements GPT suggested
(2a07090 and 5a3eb25 above) are already in.

Rollout

  • No new CI workflows or steps. SimpleCov's threshold enforcement runs as
    part of the existing bundle exec rake step in
    .github/workflows/ci.yml; any future PR that drops below 100% line or
    100% branch coverage will fail that step.
  • The CI matrix (Ruby 3.3 / 3.4 / 4.0) is unchanged. SimpleCov 0.22 only
    requires Ruby >= 2.5, so it should resolve cleanly on every matrix entry.
  • Backwards compatible: lib/github/markups.rb registers the same
    implementations in the same order. The renderer still picks the first
    match via markup_impls.find, so render precedence is unchanged.
  • After this lands I'll rebase chore(deps): bump commonmarker from v0.18.3 to v2.8.2 #2059 (the commonmarker 2.x migration) on
    top so we can use the new coverage floor as a safety net for that
    larger change.

zkoppert and others added 7 commits June 7, 2026 23:12
Replaces the bare DSL (markup, command, markup_impl) with fully-qualified
GitHub::Markup.* calls so the file can be loaded via require_relative
instead of instance_eval. This lets SimpleCov track coverage of the
registration logic.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Zack Koppert <zkoppert@github.com>
The gemspec previously `require`d "../lib/github-markup" to fetch the
VERSION constant. Bundler's gem_tasks evaluates the gemspec at rake
startup, which happens before tests load SimpleCov. Any file `require`d
from the gemspec is therefore invisible to coverage tracking.

Switch to a regex read of the version string and add simplecov as a
development dependency in preparation for the coverage target landing
in a follow-up commit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Zack Koppert <zkoppert@github.com>
The `else` branch is for RDoc < 4, which has been unsupported since
Ruby 2.4 (2016). Modern RDoc requires Options. The branch is unreachable
in any supported environment, so guard it with :nocov: markers rather
than contort the test suite to fake an unsupported RDoc version.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Zack Koppert <zkoppert@github.com>
Adds a test_helper that boots SimpleCov before the rest of the suite
loads, with line and branch coverage tracking enabled and a hard floor
at 100% for both. SimpleCov exits non-zero when the threshold isn't
met, so the existing `bundle exec rake` step in CI fails any future
change that regresses coverage.

Also adds coverage_test.rb, a focused suite that exercises the
previously uncovered code paths:

  - render fallback when no implementation matches
  - render_s nil-content guard
  - markup_impl duplicate-symbol guard
  - markdown loader (BlueCloth, Kramdown, RDiscount, Redcarpet, Maruku,
    GitHub::Markdown) with try_require fanout and final LoadError
  - Implementation base class NotImplementedError
  - CommandImplementation arity-1, arity-2, no-block, fallback-to-content,
    and non-zero-exit paths
  - language() and match? without Linguist loaded
  - Implementation constructor without Linguist + invalid language raise
  - WikiCloth ESCAPED_TAGS << 'tt' guard both first-call and repeat-call

The cat.sh fixture is the test double for the no-block command path.

Coverage: 90.23% line / 64.71% branch -> 100% line / 100% branch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Zack Koppert <zkoppert@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Zack Koppert <zkoppert@github.com>
Previously the gemspec dereferenced MatchData#[] directly, so a
formatting change in lib/github-markup.rb that broke the regex
would raise NoMethodError on nil. Wrap the match in an explicit
check and raise with the file path so the failure is obvious.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Zack Koppert <zkoppert@github.com>
The previous assertion only checked that 'tt' was present in
ESCAPED_TAGS after two renders, which would still pass if the
idempotency guard regressed and appended duplicates. Switch to
counting 'tt' occurrences so the test would fail if WikiCloth's
ESCAPED_TAGS<<'tt' ever ran twice.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Zack Koppert <zkoppert@github.com>
@zkoppert zkoppert self-assigned this Jun 8, 2026
@zkoppert zkoppert marked this pull request as ready for review June 8, 2026 06:25
Copilot AI review requested due to automatic review settings June 8, 2026 06:25
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds SimpleCov-based coverage enforcement (100% line + branch) to github/markup and introduces targeted tests to cover previously untested branches, while adjusting markup registration and gemspec version loading so coverage measurement is accurate in CI.

Changes:

  • Add SimpleCov configuration via a new test/test_helper.rb and wire it into the test suite.
  • Add a new test/coverage_test.rb with targeted tests to reach previously uncovered branches.
  • Refactor markup registration to be require_relative-loadable and adjust the gemspec to read VERSION without requiring the library (to avoid coverage blind spots).
Show a summary per file
File Description
test/test_helper.rb Starts SimpleCov with branch coverage and enforces 100% minimums.
test/markup_test.rb Requires the new test helper so coverage is enabled for existing tests.
test/fixtures/cat.sh Adds a simple command fixture used by new CommandImplementation tests.
test/coverage_test.rb Adds focused tests to cover previously untested branches and enforcement paths.
lib/github/markups.rb Switches from implicit DSL calls to explicit GitHub::Markup.* registration calls.
lib/github/markup/rdoc.rb Adds :nocov: markers for an unreachable legacy RDoc < 4 fallback.
lib/github/markup.rb Loads markups via require_relative instead of instance_eval to be visible to Coverage/SimpleCov.
github-markup.gemspec Extracts VERSION via regex instead of require-ing the library; adds SimpleCov dev dependency.
Gemfile.lock Locks SimpleCov and its transitive dependencies.
.gitignore Ignores SimpleCov’s coverage/ output directory.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 8/10 changed files
  • Comments generated: 3

Comment thread test/coverage_test.rb Outdated
Comment thread test/coverage_test.rb Outdated
Comment thread test/coverage_test.rb Outdated
zkoppert and others added 2 commits June 7, 2026 23:31
The header listed five fallback markdown gem procs, but the test file
actually exercises all six (github/markdown, redcarpet, rdiscount,
maruku, kramdown, bluecloth). The LoadError wording is now quoted
exactly, and the stale legacy RDoc < 4 bullet is removed since that
branch is excluded from coverage with :nocov: markers and no test
covers it directly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Zack Koppert <zkoppert@github.com>
The CommandImplementation empty-output test hard-coded /usr/bin/true,
which is not guaranteed to live at that path on every platform. Adding
test/fixtures/empty.sh keeps the test self-contained and matches the
existing cat.sh / fail.sh pattern, so the suite runs anywhere the rest
of the fixtures already do.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Zack Koppert <zkoppert@github.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 9/11 changed files
  • Comments generated: 0 new

@zkoppert zkoppert merged commit be33798 into master Jun 8, 2026
11 checks passed
@zkoppert zkoppert deleted the test-coverage-100 branch June 8, 2026 06:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants