Skip to content

NO-JIRA: Fix flaky storage-admin e2e test#31180

Open
jubittajohn wants to merge 1 commit into
openshift:mainfrom
jubittajohn:fix-storage-admin-flake
Open

NO-JIRA: Fix flaky storage-admin e2e test#31180
jubittajohn wants to merge 1 commit into
openshift:mainfrom
jubittajohn:fix-storage-admin-flake

Conversation

@jubittajohn
Copy link
Copy Markdown
Contributor

@jubittajohn jubittajohn commented May 14, 2026

Use a unique project name derived from the test namespace and fix the "sytem" typo in --as-group.

This failure was seen in this job run: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_kubernetes/2653/pull-ci-openshift-kubernetes-master-e2e-aws-ovn-runc/2054927169104121856

Summary by CodeRabbit

  • Tests
    • Improved CLI integration test isolation by using dynamically generated project names and ensuring reliable cleanup.

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci-robot
Copy link
Copy Markdown

@jubittajohn: This pull request explicitly references no jira issue.

Details

In response to this:

The test used a hardcoded project name "policy-can-i" which caused AlreadyExists errors on retries, and HaveSuffix("no") assertions that failed when oc auth can-i appended diagnostic messages.

Use a unique project name derived from the test namespace, switch assertions to HavePrefix, and fix the "sytem" typo in --as-group.

This failure was seen in this job run: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_kubernetes/2653/pull-ci-openshift-kubernetes-master-e2e-aws-ovn-runc/2054927169104121856

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 14, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

Walkthrough

The scoped storage-admin CLI test now generates a unique projectName (prefix policy-can-i-), uses corrected impersonation --as-group flags for oc adm new-project, defers deletion of project/<projectName>, and runs oc auth can-i --namespace=<projectName> checks with the same no/yes expectations.

Changes

storage-admin CLI permission and scoped test

Layer / File(s) Summary
Scoped admin test: dynamic project and impersonation fix
test/extended/cli/admin.go
Generate a unique projectName with policy-can-i- prefix, update oc adm new-project to pass --as-group=system:authenticated and --as-group=system:authenticated:oauth, defer deletion of project/<projectName>, and run subsequent oc auth can-i --namespace=<projectName> checks preserving the same no/yes assertions.

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • sjenning
🚥 Pre-merge checks | ✅ 10 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Test Structure And Quality ⚠️ Warning Single It block tests 5+ unrelated behaviors. Missing timeouts on cluster operations (new-project, delete). No meaningful assertion messages. Cleanup doesn't verify success. Split into separate tests, add wait.Poll timeouts to cluster ops, include failure messages in assertions, verify cleanup with Expect().
Single Node Openshift (Sno) Test Compatibility ⚠️ Warning PR adds 16 new e2e tests. The "node-logs" test queries master nodes and uses randomNode(), making node-focused assumptions. No SNO compatibility guards present (no skip labels or checks). Add [Skipped:SingleReplicaTopology] label to "node-logs" test, or guard with exutil.IsSingleNode() check. Consider applying to entire test suite for consistency.
✅ Passed checks (10 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: fixing a flaky storage-admin e2e test, which directly corresponds to the test modifications and assertions updates in the changeset.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed Test title is static. Generated projectName used only in test body, not in test title. All test names remain deterministic and stable.
Microshift Test Compatibility ✅ Passed Tests have [apigroup:authorization.openshift.io][apigroup:user.openshift.io] tags. MicroShift CI automatically skips tests with unavailable API group tags.
Topology-Aware Scheduling Compatibility ✅ Passed PR modifies test file only (test/extended/cli/admin.go). Custom check applies to deployment manifests, operator code, or controllers. No scheduling constraints or topology assumptions introduced.
Ote Binary Stdout Contract ✅ Passed Changes in g.It() test blocks (output intercepted). No process-level stdout writes detected. Complies with OTE Binary Stdout Contract.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The modified "storage-admin" test contains no IPv4 assumptions or external connectivity requirements. All operations are cluster-internal RBAC/authorization checks.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from deads2k and sjenning May 14, 2026 22:32
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/extended/cli/admin.go`:
- Around line 304-305: The projectName is deterministic (projectName :=
fmt.Sprintf("policy-can-i-%s", ocns.Namespace())) which causes AlreadyExists on
retries; change creation to append a generated unique suffix (e.g., low-entropy
random string or timestamp) to projectName before calling
oc.Run("new-project").Args(...).Execute() so each attempt uses a unique project
name; update the variable construction where projectName is defined and used
(projectName and the oc.Run("new-project") call) to include the suffix.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 2e47496d-770f-4017-a9d8-fa76a2dbe064

📥 Commits

Reviewing files that changed from the base of the PR and between b4d5333 and 7695f73.

📒 Files selected for processing (1)
  • test/extended/cli/admin.go

Comment thread test/extended/cli/admin.go Outdated
@jubittajohn jubittajohn force-pushed the fix-storage-admin-flake branch from 7695f73 to e9ad568 Compare May 14, 2026 22:41
@openshift-merge-bot openshift-merge-bot Bot added the ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review label May 14, 2026
@jubittajohn jubittajohn force-pushed the fix-storage-admin-flake branch from e9ad568 to 08a1be5 Compare May 15, 2026 03:30
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@ardaguclu
Copy link
Copy Markdown
Member

Changes look good to me
/lgtm
/hold
but we need to see the green CI results for that test

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 15, 2026
@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 15, 2026
@ardaguclu
Copy link
Copy Markdown
Member

/lgtm cancel

@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label May 15, 2026
@ardaguclu
Copy link
Copy Markdown
Member

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 15, 2026
@ardaguclu
Copy link
Copy Markdown
Member

The test used a hardcoded project name "policy-can-i" which caused
AlreadyExists errors on retries. Use a unique project name via
names.SimpleNameGenerator and fix the "sytem" typo in --as-group.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jubittajohn jubittajohn force-pushed the fix-storage-admin-flake branch from 08a1be5 to 9b7c066 Compare May 15, 2026 11:35
@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label May 15, 2026
@ardaguclu
Copy link
Copy Markdown
Member

Thanks
/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 15, 2026
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@jubittajohn
Copy link
Copy Markdown
Contributor Author

/retest

@jubittajohn
Copy link
Copy Markdown
Contributor Author

/test e2e-metal-ipi-ovn-ipv6

1 similar comment
@jubittajohn
Copy link
Copy Markdown
Contributor Author

/test e2e-metal-ipi-ovn-ipv6

Copy link
Copy Markdown
Member

@bertinatto bertinatto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 15, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ardaguclu, bertinatto, jubittajohn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 15, 2026
@jubittajohn
Copy link
Copy Markdown
Contributor Author

jubittajohn commented May 15, 2026

/test e2e-aws-ovn-serial-1of2

1 similar comment
@jubittajohn
Copy link
Copy Markdown
Contributor Author

/test e2e-aws-ovn-serial-1of2

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 16, 2026

@jubittajohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-metal-ipi-ovn-ipv6 9b7c066 link true /test e2e-metal-ipi-ovn-ipv6

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-trt
Copy link
Copy Markdown

openshift-trt Bot commented May 16, 2026

Job Failure Risk Analysis for sha: 9b7c066

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-csi Medium
verify operator conditions insights
This test has passed 85.71% of 7 runs on release 5.0 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:standard Network:ovn NetworkStack:ipv4 OS:rhcos9 Owner:eng Platform:aws Procedure:none SecurityMode:default Topology:ha Upgrade:micro] in the last week.
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 IncompleteTests
Tests for this run (20) are below the historical average (3349): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants