OLS-3238 - create ITS and pipeline for konflux periodic e2e pipeline#325
OLS-3238 - create ITS and pipeline for konflux periodic e2e pipeline#325JoaoFula wants to merge 3 commits into
Conversation
Thin wrapper around scripts/e2e-cluster.sh. Accepts optional PROVIDERS variable (defaults to 'claude gemini openai' inside the script). Ref: OLS-3238
Registers agentic-product-e2e ITS in crt-nshift-lightspeed-tenant. Triggers on operator component builds, resolves the product-e2e pipeline via git resolver. Marked optional — intended for nightly CronJob scheduling (set up out of band). Ref: OLS-3238
Provisions ephemeral Hypershift cluster, deploys operator with the real sandbox image from SNAPSHOT (falls back to :latest if sandbox component not present), then runs make product-e2e against all LLM providers (Claude, Gemini, OpenAI) with live credentials from Konflux workspace secrets (vertex-apitoken, openai). Key differences from the component-level e2e pipeline: - Mounts vertex-apitoken and openai secrets for real LLM credentials - Extracts sandbox image from SNAPSHOT (cross-component) - Calls make product-e2e (scripts/e2e-cluster.sh) instead of make test-e2e - Artifacts tagged as agentic-product-e2e-<commit> Ref: OLS-3238 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
📝 WalkthroughSummary by CodeRabbit
WalkthroughChangesAgentic product E2E
Sequence Diagram(s)sequenceDiagram
participant Scenario
participant TektonPipeline
participant OpenShift
participant E2ETests
participant Quay
Scenario->>TektonPipeline: resolve and start product E2E pipeline
TektonPipeline->>OpenShift: provision cluster and install operator
TektonPipeline->>E2ETests: run tests with provider credentials
E2ETests-->>TektonPipeline: produce logs and test artifacts
TektonPipeline->>Quay: publish collected artifacts
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
@JoaoFula: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (4)
.tekton/integration-tests/pipelines/agentic-product-e2e-pipeline.yaml (4)
2-3: 📐 Maintainability & Code Quality | 🔵 Trivial
tekton.dev/v1beta1is deprecated.
Pipeline/Taskv1beta1has been deprecated since Tekton Pipelines v0.50.0 in favor ofv1. Since this is a brand-new file, prefertekton.dev/v1.Please confirm the cluster's Tekton Pipelines version still supports
v1beta1before merging, or migrate directly tov1(field names are otherwise compatible for this pipeline's usage).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.tekton/integration-tests/pipelines/agentic-product-e2e-pipeline.yaml around lines 2 - 3, Update the Pipeline manifest’s apiVersion from tekton.dev/v1beta1 to tekton.dev/v1, preserving the existing Pipeline specification because its fields are compatible with v1.
182-204: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win
install-operatorhas noonError: continue, unlike the surrounding steps.If operator installation fails,
gather-cluster-resources/push-artifacts/fail-if-any-step-failednever run, losing cluster diagnostics for install-time failures (kubeconfig is already available at this point).♻️ Allow diagnostics to run even on install failure
- name: install-operator + onError: continue volumeMounts:🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.tekton/integration-tests/pipelines/agentic-product-e2e-pipeline.yaml around lines 182 - 204, Add `onError: continue` to the `install-operator` step so operator installation failures do not stop the pipeline. Preserve the existing diagnostic and failure-reporting steps—`gather-cluster-resources`, `push-artifacts`, and `fail-if-any-step-failed`—so they still run and report the installation failure.
254-266: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCluster-resource gathering never captures the
AgenticRunresource itself.Sub-resources (proposals, analysisresults, executionresults, verificationresults) are collected, but the primary
AgenticRun— whose phase/conditions driveDerivePhase()— isn't dumped, making it harder to triage failures.As per coding guidelines, `**`: "Run lifecycle phases... Derived from conditions via `DerivePhase()` — never stored on the spec", meaning the `AgenticRun` conditions are the primary triage source.♻️ Add AgenticRun capture
oc get proposals -A -o yaml > proposals.yaml 2>/dev/null || true + oc get agenticruns -A -o yaml > agenticruns.yaml 2>/dev/null || true oc get proposalapprovals -A -o yaml > proposalapprovals.yaml 2>/dev/null || true🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.tekton/integration-tests/pipelines/agentic-product-e2e-pipeline.yaml around lines 254 - 266, Update the cluster-resource gathering script to dump the AgenticRun resources alongside the existing proposals, analysisresults, executionresults, and verificationresults captures. Use oc get for the AgenticRun resource across all namespaces and write the YAML output to a clearly named artifact file, preserving the existing error-tolerant behavior.Source: Coding guidelines
149-149: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMultiple steps pin base images to
:latest.
registry.redhat.io/openshift4/ose-cli:latest,quay.io/konflux-qe-incubator/konflux-qe-tools:latest, and the sandbox fallback all float onlatest, which makes nightly runs non-reproducible if the upstream image changes unexpectedly.Pin these to specific tags/digests for reproducible CI runs.
Also applies to: 171-171, 191-191, 234-234, 253-253, 269-269
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.tekton/integration-tests/pipelines/agentic-product-e2e-pipeline.yaml at line 149, Replace every floating :latest image reference in the agentic product E2E pipeline, including the ose-cli, konflux-qe-tools, and sandbox fallback images identified by the affected steps, with approved immutable version tags or digests. Keep each step’s existing image purpose unchanged while ensuring all referenced images are reproducible.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In @.tekton/integration-tests/pipelines/agentic-product-e2e-pipeline.yaml:
- Around line 2-3: Update the Pipeline manifest’s apiVersion from
tekton.dev/v1beta1 to tekton.dev/v1, preserving the existing Pipeline
specification because its fields are compatible with v1.
- Around line 182-204: Add `onError: continue` to the `install-operator` step so
operator installation failures do not stop the pipeline. Preserve the existing
diagnostic and failure-reporting steps—`gather-cluster-resources`,
`push-artifacts`, and `fail-if-any-step-failed`—so they still run and report the
installation failure.
- Around line 254-266: Update the cluster-resource gathering script to dump the
AgenticRun resources alongside the existing proposals, analysisresults,
executionresults, and verificationresults captures. Use oc get for the
AgenticRun resource across all namespaces and write the YAML output to a clearly
named artifact file, preserving the existing error-tolerant behavior.
- Line 149: Replace every floating :latest image reference in the agentic
product E2E pipeline, including the ose-cli, konflux-qe-tools, and sandbox
fallback images identified by the affected steps, with approved immutable
version tags or digests. Keep each step’s existing image purpose unchanged while
ensuring all referenced images are reproducible.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 3c46efb9-ccf3-45db-9a06-1ae744ea4776
📒 Files selected for processing (3)
.tekton/integration-tests/pipelines/agentic-product-e2e-pipeline.yaml.tekton/integration-tests/product-e2e-test-scenario.yamlMakefile
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
openshift/lightspeed-agentic-sandbox(manual)
1 participant
No description provided.