Skip to content

feat(integration): Go-native runner for YAML integration tests (POC)#3537

Open
amir-deris wants to merge 8 commits into
mainfrom
amir/plt-429-go-runner-for-integration-tests-yaml
Open

feat(integration): Go-native runner for YAML integration tests (POC)#3537
amir-deris wants to merge 8 commits into
mainfrom
amir/plt-429-go-runner-for-integration-tests-yaml

Conversation

@amir-deris
Copy link
Copy Markdown
Contributor

@amir-deris amir-deris commented Jun 3, 2026

Summary

  • Adds integration_test/runner/ — a Go-native driver for the existing YAML integration test suite that replaces the Python runner.py for covered modules
  • Keeps all existing YAML test files unchanged; the runner parses the same schema
  • Wires one CI matrix job ("Mint & Staking & Bank Module") to the Go runner as a proof-of-concept, leaving the parallel Autobahn job on Python for side-by-side comparison

Motivation

The Python runner has a few pain points:

  • Failures stop the entire file (exit(1)) with no subtest isolation
  • No -run flag to target a single test case without editing files
  • Output is flat print() statements with no structured test reporting
  • Python dependency in a Go project's CI

The Go runner addresses all of these through standard go test integration.

How it works

Each YAML file maps to a t.Run subtest, so failures are isolated and the full -v output shows every command and its captured output. The eval verifier uses big.Int → float64 → string comparison (in that order) to handle the large token supply integers in the existing YAML assertions without float64 precision loss.

Run a single module locally (requires a running cluster):

go test -tags yaml_integration -v ./integration_test/runner/... -run TestBankModule

Target a single test case:

go test -tags yaml_integration -v ./integration_test/runner/... -run TestBankModule/Test_sending_funds

@cursor
Copy link
Copy Markdown

cursor Bot commented Jun 3, 2026

PR Summary

Low Risk
Test harness and CI wiring only; YAML scenarios unchanged and Autobahn still exercises the Python path.

Overview
Introduces a Go-native YAML integration test runner (integration_test/runner/) that drives the same YAML files as runner.py through standard go test, with each YAML case as a named subtest (-run filtering, per-case failures, verbose command logs).

The runner unmarshals inputs/verifiers, runs commands via docker exec (default sei-node-0) or local bash, and implements eval / regex verifiers with Python-compatible logic—including big.Int comparisons for large supply integers. gopkg.in/yaml.v3 is promoted to a direct module dependency.

CI POC: the Mint & Staking & Bank Module matrix job now runs go test -tags yaml_integration for TestMintModule, TestStakingModule, and TestBankModule/Test_sending_funds instead of three separate runner.py invocations; the parallel Autobahn job still uses Python for side-by-side comparison.

Reviewed by Cursor Bugbot for commit cac538b. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 3, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedJun 3, 2026, 1:40 PM

@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 3, 2026

Codecov Report

❌ Patch coverage is 0% with 176 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.22%. Comparing base (2151ba0) to head (cac538b).

Files with missing lines Patch % Lines
integration_test/runner/runner.go 0.00% 176 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3537      +/-   ##
==========================================
- Coverage   59.12%   58.22%   -0.91%     
==========================================
  Files        2213     2141      -72     
  Lines      182814   174607    -8207     
==========================================
- Hits       108096   101660    -6436     
+ Misses      64993    63901    -1092     
+ Partials     9725     9046     -679     
Flag Coverage Δ
sei-chain-pr 0.00% <0.00%> (?)
sei-db 70.41% <ø> (ø)
sei-db-state-db ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
integration_test/runner/runner.go 0.00% <0.00%> (ø)

... and 73 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread integration_test/runner/runner.go
Comment thread integration_test/runner/runner_test.go
@amir-deris amir-deris changed the title Added go runner for yaml tests feat(integration): Go-native runner for YAML integration tests (POC) Jun 3, 2026
Comment thread .github/workflows/integration-test.yml Outdated
@amir-deris amir-deris requested review from bdchatham and masih June 3, 2026 12:08
- Run go mod tidy to move gopkg.in/yaml.v3 from indirect to direct deps
- Fix shell quoting in CI YAML: replace single quotes in -run regex with escaped double quotes to avoid breaking the echo '...' wrapper in the test runner script

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4ddd20c. Configure here.

Comment thread .github/workflows/integration-test.yml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant