[Debug] Test e2-standard-8 control plane for gcp-ovn-rt-upgrade by sdodson · Pull Request #82754 · openshift/release

sdodson · 2026-07-31T18:41:58Z

Summary

Overrides CONTROL_PLANE_NODE_TYPE from the default e2-custom-6-16384 to e2-standard-8 for e2e-gcp-ovn-rt-upgrade.
Purpose: test whether the extra CPU headroom (8 vs 6 vCPU) and higher persistent-disk throughput ceiling (800 vs 240 MiB/s) reduce the etcd disk-latency SLO breaches and control-plane CPU saturation found in prior investigation of this job's flakiness (see the case *) comment in ipi-conf-gcp-commands.sh noting e2-custom-6-16384 was already a "temporary test" for the same symptom).
Will trigger a rehearsal to collect comparison data (etcd fsync/commit latency, CPU p90) against the current default.

⚠️ DO NOT MERGE — rehearsal-only experiment, will be closed after data collection.

Summary by CodeRabbit

The e2e-gcp-ovn-rt-upgrade OpenShift CI rehearsal will use e2-custom-8-16384 control-plane nodes.
The rehearsal evaluates whether eight vCPUs with 16 GB memory reduce etcd latency SLO breaches and control-plane CPU saturation.
The change is for testing only and must not be merged.

Overrides CONTROL_PLANE_NODE_TYPE from the default e2-custom-6-16384 to e2-standard-8 to test whether the extra CPU headroom (8 vs 6 vCPU) and higher persistent-disk throughput ceiling (800 vs 240 MiB/s) reduce the etcd disk-latency SLO breaches and control-plane CPU saturation found in prior investigation of this job's flakiness. DO NOT MERGE - rehearsal-only experiment, will be closed after data collection. rh-pre-commit.version: 2.4.0 rh-pre-commit.check-secrets: ENABLED

openshift-ci · 2026-07-31T18:42:24Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sdodson

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~ci-operator/config/openshift/release/OWNERS~~ [sdodson]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

coderabbitai · 2026-07-31T18:42:43Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 66eb365b-02c7-486f-a4e0-3a1968190f6c

📥 Commits

Reviewing files that changed from the base of the PR and between e29c8a2 and 301919f.

📒 Files selected for processing (1)

ci-operator/config/openshift/release/openshift-release-main__ci-5.0-upgrade-from-stable-4.22.yaml

Walkthrough

The GCP OVN realtime upgrade test now sets CONTROL_PLANE_NODE_TYPE to e2-custom-8-16384.

Changes

GCP OVN upgrade test

Layer / File(s)	Summary
Configure control plane node type `ci-operator/config/openshift/release/openshift-release-main__ci-5.0-upgrade-from-stable-4.22.yaml`	The test environment sets `CONTROL_PLANE_NODE_TYPE` to `e2-custom-8-16384`.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Suggested labels: rehearsals-ack

Suggested reviewers: sosiouxme, xueqzhan

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Title check	⚠️ Warning	The title names `e2-standard-8`, but the change sets `CONTROL_PLANE_NODE_TYPE` to `e2-custom-8-16384`.	Update the title to reference `e2-custom-8-16384` or change the configuration to match the stated `e2-standard-8` test.

✅ Passed checks (14 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The patch changes only CONTROL_PLANE_NODE_TYPE; it adds no Ginkgo title and the existing e2e-gcp-ovn-rt-upgrade identifier is static.
Test Structure And Quality	✅ Passed	The PR changes only one YAML environment value; it adds no Ginkgo It blocks or test source to review for setup, waits, assertions, or cleanup.
Microshift Test Compatibility	✅ Passed	The PR changes only a CI YAML node-type value; it adds no Ginkgo tests or test declarations that require MicroShift compatibility review.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	The commit changes only the existing YAML job's CONTROL_PLANE_NODE_TYPE; it adds no Ginkgo It, Describe, Context, or When test, so the SNO check does not apply.
Topology-Aware Scheduling Compatibility	✅ Passed	The only change is a CI test environment value, CONTROL_PLANE_NODE_TYPE; it adds no deployment manifest, operator code, controller, or scheduling constraint.
Ote Binary Stdout Contract	✅ Passed	The PR changes only one YAML environment value; it adds no OTE binary or process-level Go stdout code, so the stdout contract check is not applicable.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	The patch changes only a YAML test configuration value; it adds no Ginkgo test, IPv4 assumption, or external connectivity requirement.
No-Weak-Crypto	✅ Passed	The commit only changes a YAML node-type value; no MD5, SHA1, DES, RC4, Blowfish, ECB, custom crypto, or secret comparison is introduced.
Container-Privileges	✅ Passed	The only code change sets CONTROL_PLANE_NODE_TYPE; the changed manifest and referenced workflow contain no privileged, host namespace, SYS_ADMIN, or escalation settings.
No-Sensitive-Data-In-Logs	✅ Passed	The only changed line sets CONTROL_PLANE_NODE_TYPE to the non-sensitive machine type e2-custom-8-16384; no logging or credential, PII, hostname, or customer data was added.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

sdodson · 2026-07-31T18:45:46Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-07-31T18:45:48Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

sdodson · 2026-08-01T00:27:49Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-08-01T00:27:52Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

sdodson · 2026-08-01T05:49:43Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-08-01T05:49:46Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

sdodson · 2026-08-01T11:19:45Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-08-01T11:19:48Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

Switches CONTROL_PLANE_NODE_TYPE from e2-standard-8 (which showed strong improvement across 4 rehearsal runs) to n2-standard-4, to test the low end of the sizing curve. Expected to perform worse than the current e2-custom-6-16384 baseline: n2-standard-4 shares the same disk I/O ceiling (2-7 vCPU tier: 15,000 IOPS / 240 MiB/s) but has less CPU headroom (4 vCPU) than the already CPU-constrained 6 vCPU baseline. Running anyway to confirm the CPU-headroom theory from the other direction and establish the low end of the cost/performance curve. DO NOT MERGE - rehearsal-only experiment. rh-pre-commit.version: 2.4.0 rh-pre-commit.check-secrets: ENABLED

sdodson · 2026-08-01T16:35:15Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-08-01T16:35:18Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

sdodson · 2026-08-01T22:03:06Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-08-01T22:03:09Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

sdodson · 2026-08-02T03:05:39Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-08-02T03:05:41Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

sdodson · 2026-08-02T03:31:44Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-08-02T03:31:46Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

sdodson · 2026-08-02T09:07:02Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-08-02T09:07:04Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

Third series in the control-plane sizing campaign. n2-standard-4 (4-run avg) performed clearly worse than the e2-custom-6-16384 baseline across CPU saturation and etcd disk-latency metrics (2/4 runs failed), confirming the CPU-headroom theory from the low end. e2-standard-8 (previous series, 4/4 passed) performed clearly better. This series tests e2-custom-8-16384: same 8 vCPU as e2-standard-8 but keeping the 16GB memory ratio matching the current e2-custom-6-16384 baseline, to isolate whether the memory ratio matters independent of vCPU count. DO NOT MERGE - rehearsal-only experiment. rh-pre-commit.version: 2.4.0 rh-pre-commit.check-secrets: ENABLED

sdodson · 2026-08-02T14:38:37Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-08-02T14:38:40Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

openshift-merge-bot · 2026-08-02T14:41:49Z

[REHEARSALNOTIFIER]
@sdodson: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade	N/A	periodic	Ci-operator config changed

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

sdodson · 2026-08-02T20:07:15Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-08-02T20:07:17Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

sdodson · 2026-08-03T01:38:05Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-08-03T01:38:08Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

openshift-ci · 2026-08-03T06:28:42Z

@sdodson: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/rehearse/periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade	`301919f`	link	unknown	`/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

sdodson · 2026-08-03T06:37:12Z

/pj-rehearse periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

openshift-merge-bot · 2026-08-03T06:37:15Z

@sdodson: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 31, 2026

openshift-ci Bot requested review from sosiouxme and xueqzhan July 31, 2026 18:42

Uh oh!

Conversation

sdodson commented Jul 31, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Summary by CodeRabbit

Uh oh!

openshift-ci Bot commented Jul 31, 2026

Uh oh!

coderabbitai Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

sdodson commented Jul 31, 2026

Uh oh!

openshift-merge-bot Bot commented Jul 31, 2026

Uh oh!

sdodson commented Aug 1, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 1, 2026

Uh oh!

sdodson commented Aug 1, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 1, 2026

Uh oh!

sdodson commented Aug 1, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 1, 2026

Uh oh!

sdodson commented Aug 1, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 1, 2026

Uh oh!

sdodson commented Aug 1, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 1, 2026

Uh oh!

sdodson commented Aug 2, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 2, 2026

Uh oh!

sdodson commented Aug 2, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 2, 2026

Uh oh!

sdodson commented Aug 2, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 2, 2026

Uh oh!

sdodson commented Aug 2, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 2, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 2, 2026

Uh oh!

sdodson commented Aug 2, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 2, 2026

Uh oh!

sdodson commented Aug 3, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 3, 2026

Uh oh!

openshift-ci Bot commented Aug 3, 2026

Uh oh!

sdodson commented Aug 3, 2026

Uh oh!

openshift-merge-bot Bot commented Aug 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

sdodson commented Jul 31, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading