Skip to content

ROSAENG-61412 | fix: align y-stream upgrade tests with channel flow#3353

Open
olucasfreitas wants to merge 1 commit into
masterfrom
ROSAENG-61412/fix-y-stream-upgrade-flow
Open

ROSAENG-61412 | fix: align y-stream upgrade tests with channel flow#3353
olucasfreitas wants to merge 1 commit into
masterfrom
ROSAENG-61412/fix-y-stream-upgrade-flow

Conversation

@olucasfreitas

@olucasfreitas olucasfreitas commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Add shared y-stream upgrade preparation helpers that compute and apply the next-minor channel, then wait for a next-minor upgrade target to appear before the tests proceed.

This aligns the e2e upgrade cases with the documented ROSA upgrade flow, removes duplicated channel setup, and keeps dedicated upgrade lanes red when no real upgrade path can be exercised.

Supersedes #3343.

Detailed Description of the Issue

Jira: ROSAENG-61412

The previous change in #3343 addressed the symptom (rosa list upgrade returning an empty list) by skipping the dedicated non-STS upgrade lane. Amanda correctly pointed out that this would let the lane go green without actually exercising an upgrade.

The deeper issue is that the y-stream cases were not consistently following the documented ROSA flow for minor upgrades: the cluster needs to move to the next-minor channel before the tests ask for available upgrade targets. Some cases changed the channel first, others did not, and the duplicated logic also skipped the channel update entirely when the cluster reported an empty channel.

This PR introduces a shared preparation flow that:

  • computes the desired y+1 channel from the profile channel group and cluster raw version,
  • applies that channel even when the current channel is empty,
  • waits for a next-minor target to appear from rosa list upgrade, and
  • fails with diagnostics if no target appears after bounded waiting.

Type of Change

  • fix - resolves an incorrect behavior or bug.

Previous Behavior

Y-stream upgrade tests had inconsistent setup:

  • some tests queried rosa list upgrade directly and assumed targets were already available,
  • some tests changed the channel first, but only if CD.Channel != "",
  • and the dedicated non-STS lane could be made to skip instead of validating a real upgrade path.

Behavior After This Change

All affected y-stream upgrade cases now use one shared preparation flow before listing or scheduling upgrades. Dedicated upgrade lanes remain red when they cannot discover a real next-minor target after channel preparation, while shared setup logic is centralized and covered by focused tests.

Testing

  • make fmt
  • make fmt-check
  • make lint
  • make test
  • make rosa
  • GOTOOLCHAIN=auto go test ./tests/e2e/... -run TestDoesNotExist -count=0
  • GOTOOLCHAIN=auto go test ./tests/utils/exec/rosacli/... -run TestDoesNotExist -count=0

Summary by CodeRabbit

  • New Features
    • Added end-to-end Y-stream upgrade discovery with a centralized prepare-and-wait flow for the next-minor channel and matching upgrade version.
  • Bug Fixes
    • Reduced flaky upgrade behavior by unifying Y-stream target selection across scheduled/dry-run and wide AMI upgrade paths.
  • Tests
    • Updated multiple upgrade E2E scenarios and expanded unit test coverage for next-minor selection, polling/timeout behavior, and input validation.
  • Documentation / Chores
    • Updated govulncheck guidance to handle missing jq, and enhanced the govulncheck helper to download/verify a pinned jq binary.

@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 4b2c7d9b-2f63-40b8-8f85-bc69f964901c

📥 Commits

Reviewing files that changed from the base of the PR and between 0c98f2a and 519ffdd.

📒 Files selected for processing (8)
  • CONTRIBUTING.md
  • hack/govulncheck.sh
  • renovate.json
  • tests/e2e/test_rosacli_upgrade.go
  • tests/utils/exec/rosacli/cluster_service.go
  • tests/utils/exec/rosacli/cluster_service_test.go
  • tests/utils/exec/rosacli/upgrade_service.go
  • tests/utils/exec/rosacli/upgrade_service_test.go
✅ Files skipped from review due to trivial changes (1)
  • CONTRIBUTING.md
🚧 Files skipped from review as they are similar to previous changes (5)
  • tests/utils/exec/rosacli/cluster_service_test.go
  • tests/utils/exec/rosacli/upgrade_service_test.go
  • tests/utils/exec/rosacli/cluster_service.go
  • tests/e2e/test_rosacli_upgrade.go
  • tests/utils/exec/rosacli/upgrade_service.go

📝 Walkthrough

Walkthrough

This PR adds y-stream upgrade preparation and next-minor upgrade discovery helpers for ROSA CLI tests, then updates multiple e2e upgrade flows to use the shared discovery path. It also adds jq bootstrap and checksum verification to hack/govulncheck.sh, updates the related documentation, and configures Renovate to track the jq version used by that script.

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Microshift Test Compatibility ⚠️ Warning New e2e upgrade suites use rosa upgrade/list/describe scheduling with no MicroShift tag or runtime guard, and upgrade workflows are unsupported on MicroShift. Add [Skipped:MicroShift] or an IsMicroShiftCluster()+g.Skip() guard (or an appropriate apigroup tag) to the affected e2e upgrade tests.
✅ Passed checks (14 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise, specific, and matches the main change: aligning y-stream upgrade tests with channel flow.
Description check ✅ Passed The description covers the issue, fix, impact, and testing, and is mostly complete despite some template sections being abbreviated or missing.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The added/changed Ginkgo titles are static strings; no dynamic names, timestamps, UUIDs, node/namespace/IP values, or generated suffixes were introduced.
Test Structure And Quality ✅ Passed New tests are focused, use cleanup for temp artifacts, and the e2e cases keep bounded setup/teardown; no clear timeout or multi-responsibility regression.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The added e2e upgrade tests only exercise ROSA CLI upgrade/list/describe flows and don’t count or drain nodes, spread pods, or assume multi-node HA.
Topology-Aware Scheduling Compatibility ✅ Passed No deployment manifests, operator code, or controllers were changed; the PR only updates tests/helpers/docs/scripts and adds no scheduling constraints.
Ote Binary Stdout Contract ✅ Passed The only stdout write found is fmt.Println inside an It block; no init/TestMain/BeforeSuite/RunSpecs stdout writes appear in the edited files.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed Changed e2e tests contain no hardcoded IPs, ParseIP/CIDR, To4, or public URLs; they only use ROSA/cluster API flows.
No-Weak-Crypto ✅ Passed Touched code only adds SHA-256 checksum verification and version/channel comparisons; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB or secret comparisons found.
Container-Privileges ✅ Passed No changed file is a container/K8s manifest, and the diff contains no privileged, hostPID/hostNetwork/hostIPC, SYS_ADMIN, or allowPrivilegeEscalation settings.
No-Sensitive-Data-In-Logs ✅ Passed No new logs/prints expose secrets or PII; added output only reports versions, channels, and generic jq/bootstrap errors.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ROSAENG-61412/fix-y-stream-upgrade-flow

Comment @coderabbitai help to get the list of available commands.

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 8, 2026
@olucasfreitas

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (7)
tests/utils/exec/rosacli/cluster_service.go (1)

53-65: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Add doc comments to new exported symbols.

YStreamUpgradePreparation and the new PrepareClusterForYStreamUpgrade interface method are exported but have no doc comments.

As per coding guidelines, "Use exported symbol doc comments when new public types or functions are introduced."

📝 Proposed doc comments
+	// PrepareClusterForYStreamUpgrade computes the next-minor channel for the given
+	// channel group and applies it to the cluster if not already set.
 	PrepareClusterForYStreamUpgrade(clusterID string, channelGroup string) (*YStreamUpgradePreparation, error)
 }
 
 type clusterService struct {
 	ResourcesService
 }
 
+// YStreamUpgradePreparation captures the channel information computed and applied
+// while preparing a cluster for a y-stream (next-minor) upgrade.
 type YStreamUpgradePreparation struct {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/utils/exec/rosacli/cluster_service.go` around lines 53 - 65, Add doc
comments for the new exported symbols in the cluster service API: document the
PrepareClusterForYStreamUpgrade interface method on the service interface and
the YStreamUpgradePreparation type definition. Keep the comments concise and
descriptive, and make sure they clearly explain the purpose of the method and
the structure so they satisfy exported symbol documentation guidelines.

Source: Coding guidelines

tests/utils/exec/rosacli/upgrade_service_test.go (1)

12-106: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add tests for the remaining validation/error branches.

FindNextMinorUpgrade's parse-error branches, and waitForAvailableYStreamUpgrade's clusterID/preparation nil-guards and non-positive interval/timeout fallback, are untested. These are part of the changed lines subject to the repo's coverage gate.

As per coding guidelines, "All code should be covered by tests, and Go tests in this repository must use Ginkgo."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/utils/exec/rosacli/upgrade_service_test.go` around lines 12 - 106, Add
Ginkgo coverage for the missing validation and error branches in
FindNextMinorUpgrade and waitForAvailableYStreamUpgrade. Create tests that
exercise the version parse failures in UpgradeVersionList.FindNextMinorUpgrade,
plus the nil-guard paths for clusterID and YStreamUpgradePreparation, and the
fallback behavior when timeout/interval inputs are non-positive. Use the
existing helper symbols in upgrade_service_test.go so the new cases are easy to
locate and keep the assertions focused on the returned errors and fallback
values.

Source: Coding guidelines

tests/utils/exec/rosacli/cluster_service_test.go (1)

17-74: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add tests for the remaining error branches in prepareYStreamUpgradeChannel.

Only the happy paths are tested. The editCluster == nil error branch, the editCluster returning an error branch, and the empty-channelGroup validation in computeNextMinorChannel are untested. Given this repo enforces an 80% changed-line coverage gate, these missing branches risk failing coverage-changed-files.

As per coding guidelines, "All code should be covered by tests, and Go tests in this repository must use Ginkgo."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/utils/exec/rosacli/cluster_service_test.go` around lines 17 - 74, The
current Ginkgo coverage in cluster_service_test.go only exercises happy paths
for computeNextMinorChannel and prepareYStreamUpgradeChannel; add tests for the
missing error branches. Cover computeNextMinorChannel rejecting an empty
channelGroup, and add cases in the prepareYStreamUpgradeChannel tests that
verify the nil editCluster path and the editCluster-returned-error path, using
the existing helper names prepareYStreamUpgradeChannel and
computeNextMinorChannel so the new specs stay aligned with the implementation.

Source: Coding guidelines

tests/e2e/test_rosacli_upgrade.go (2)

711-719: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Hoist the repeated clusterService declaration into the shared setup.

clusterService := rosaClient.Cluster is now declared inline in two separate It blocks within this Describe. Since upgradeService is already promoted to the shared var/BeforeEach for this block, doing the same for clusterService would avoid the duplicated boilerplate.

Also applies to: 788-795

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/e2e/test_rosacli_upgrade.go` around lines 711 - 719, Hoist the repeated
clusterService setup into the shared Describe-level state by declaring
clusterService alongside upgradeService and initializing it in the shared
BeforeEach, then update the affected It blocks to use that shared variable
instead of re-declaring rosaClient.Cluster. Keep the same pattern used for
upgradeService so the prepareYStreamUpgradeVersion calls in both test cases
reference the shared clusterService consistently.

349-361: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Stale "decide if skip this case" messaging no longer matches fail-fast behavior.

These By(...) steps still say "to decide if skip this case," but prepareYStreamUpgradeVersion now fails the test via Expect(err).ToNot(HaveOccurred()) rather than skipping when no upgrade target is found — consistent with the PR's stated intent to keep these lanes red. The stale wording could confuse CI failure triage. Other call sites in this same PR (e.g., line 437, 537) were updated to "Prepare cluster for y-stream upgrade target discovery."

✏️ Suggested wording fix
-			By("Check the cluster upgrade version to decide if skip this case")
+			By("Prepare cluster for y-stream upgrade target discovery")
 			upgradingVersion, err := prepareYStreamUpgradeVersion(

Also applies to: 378-389

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/e2e/test_rosacli_upgrade.go` around lines 349 - 361, Update the stale
By step messaging in the upgrade test to match the fail-fast behavior now used
by prepareYStreamUpgradeVersion. In test_rosacli_upgrade.go, change the
descriptive text around the calls to prepareYStreamUpgradeVersion and
Expect(err).ToNot(HaveOccurred()) from “decide if skip this case” to the new
“Prepare cluster for y-stream upgrade target discovery” wording used elsewhere
in the same test, so the intent is consistent at the call sites around the
upgrade flow.
tests/utils/exec/rosacli/upgrade_service.go (2)

179-184: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Extract the fallback wait interval/timeout as a named constant.

time.Second is used as a silent fallback when waitInterval/waitTimeout are non-positive. Extracting this into a named constant (e.g., defaultYStreamPollInterval) would make the fallback behavior self-documenting and avoid a bare magic literal.

As per coding guidelines, "Avoid magic numbers; extract named constants when the value matters to behavior or readability."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/utils/exec/rosacli/upgrade_service.go` around lines 179 - 184, The
fallback behavior in the wait interval/timeout handling uses a bare time.Second
literal, so extract that value into a named constant and use it in the existing
waitInterval and waitTimeout checks. Update the logic in the relevant helper to
reference the new constant (for example, defaultYStreamPollInterval) so the
default polling behavior is self-documenting and easier to maintain.

Source: Coding guidelines


165-237: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Consider context.Context for the polling/timeout loop instead of manual deadline tracking.

waitForAvailableYStreamUpgrade implements its own deadline+sleep loop. Using context.Context with context.WithTimeout (and ctx.Done()/ctx.Err()) is the idiomatic Go pattern for cancellation and timeouts and would let callers cancel the wait early (e.g., on test suite interruption) instead of always blocking to the fixed deadline.

As per coding guidelines, Go security requirements call for "context.Context for cancellation and timeouts."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/utils/exec/rosacli/upgrade_service.go` around lines 165 - 237,
`waitForAvailableYStreamUpgrade` currently uses a manual deadline and sleep loop
for polling, which should be replaced with `context.Context`-based cancellation
and timeout handling. Update the function to accept a context (or derive one
with timeout from the caller), use `context.WithTimeout` plus `ctx.Done()` in
the loop instead of `time.Now().After`/`time.Sleep`, and return promptly on
cancellation via `ctx.Err()` while keeping the existing fetch/version lookup
logic intact.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/e2e/test_rosacli_upgrade.go`:
- Around line 711-719: Hoist the repeated clusterService setup into the shared
Describe-level state by declaring clusterService alongside upgradeService and
initializing it in the shared BeforeEach, then update the affected It blocks to
use that shared variable instead of re-declaring rosaClient.Cluster. Keep the
same pattern used for upgradeService so the prepareYStreamUpgradeVersion calls
in both test cases reference the shared clusterService consistently.
- Around line 349-361: Update the stale By step messaging in the upgrade test to
match the fail-fast behavior now used by prepareYStreamUpgradeVersion. In
test_rosacli_upgrade.go, change the descriptive text around the calls to
prepareYStreamUpgradeVersion and Expect(err).ToNot(HaveOccurred()) from “decide
if skip this case” to the new “Prepare cluster for y-stream upgrade target
discovery” wording used elsewhere in the same test, so the intent is consistent
at the call sites around the upgrade flow.

In `@tests/utils/exec/rosacli/cluster_service_test.go`:
- Around line 17-74: The current Ginkgo coverage in cluster_service_test.go only
exercises happy paths for computeNextMinorChannel and
prepareYStreamUpgradeChannel; add tests for the missing error branches. Cover
computeNextMinorChannel rejecting an empty channelGroup, and add cases in the
prepareYStreamUpgradeChannel tests that verify the nil editCluster path and the
editCluster-returned-error path, using the existing helper names
prepareYStreamUpgradeChannel and computeNextMinorChannel so the new specs stay
aligned with the implementation.

In `@tests/utils/exec/rosacli/cluster_service.go`:
- Around line 53-65: Add doc comments for the new exported symbols in the
cluster service API: document the PrepareClusterForYStreamUpgrade interface
method on the service interface and the YStreamUpgradePreparation type
definition. Keep the comments concise and descriptive, and make sure they
clearly explain the purpose of the method and the structure so they satisfy
exported symbol documentation guidelines.

In `@tests/utils/exec/rosacli/upgrade_service_test.go`:
- Around line 12-106: Add Ginkgo coverage for the missing validation and error
branches in FindNextMinorUpgrade and waitForAvailableYStreamUpgrade. Create
tests that exercise the version parse failures in
UpgradeVersionList.FindNextMinorUpgrade, plus the nil-guard paths for clusterID
and YStreamUpgradePreparation, and the fallback behavior when timeout/interval
inputs are non-positive. Use the existing helper symbols in
upgrade_service_test.go so the new cases are easy to locate and keep the
assertions focused on the returned errors and fallback values.

In `@tests/utils/exec/rosacli/upgrade_service.go`:
- Around line 179-184: The fallback behavior in the wait interval/timeout
handling uses a bare time.Second literal, so extract that value into a named
constant and use it in the existing waitInterval and waitTimeout checks. Update
the logic in the relevant helper to reference the new constant (for example,
defaultYStreamPollInterval) so the default polling behavior is self-documenting
and easier to maintain.
- Around line 165-237: `waitForAvailableYStreamUpgrade` currently uses a manual
deadline and sleep loop for polling, which should be replaced with
`context.Context`-based cancellation and timeout handling. Update the function
to accept a context (or derive one with timeout from the caller), use
`context.WithTimeout` plus `ctx.Done()` in the loop instead of
`time.Now().After`/`time.Sleep`, and return promptly on cancellation via
`ctx.Err()` while keeping the existing fetch/version lookup logic intact.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 9210d02d-e098-4b47-93c6-5033b98a4a49

📥 Commits

Reviewing files that changed from the base of the PR and between 6fba0b4 and 0c98f2a.

📒 Files selected for processing (5)
  • tests/e2e/test_rosacli_upgrade.go
  • tests/utils/exec/rosacli/cluster_service.go
  • tests/utils/exec/rosacli/cluster_service_test.go
  • tests/utils/exec/rosacli/upgrade_service.go
  • tests/utils/exec/rosacli/upgrade_service_test.go

Add shared y-stream upgrade preparation helpers that compute and apply
the next-minor channel, then wait for a next-minor upgrade target to
appear before the tests proceed.

This aligns the e2e upgrade cases with the documented ROSA upgrade flow,
removes duplicated channel setup, and keeps dedicated upgrade lanes red
when no real upgrade path can be exercised.
@olucasfreitas olucasfreitas force-pushed the ROSAENG-61412/fix-y-stream-upgrade-flow branch from 9bdaad3 to 519ffdd Compare July 8, 2026 16:38
@openshift-ci

openshift-ci Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: olucasfreitas

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci

openshift-ci Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

@olucasfreitas: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant