Add a dashboard for Compliance Operator

[xiaojiey]

Previously, I created a PR #3137 to create a dashboard for Compliance Operator. However, the release cycle of Compliance Operator doesn't match with OpenShift. The base date means nothing. Create a new dashboard to show the job results.

[openshift-ci-robot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

[coderabbitai]

Walkthrough

Adds a new top-level compliance-operator section under releases in the OpenShift customizations config, introducing a nested jobs mapping that lists numerous periodic job entries for upstream ComplianceAsCode and downstream compliance-operator workflows.

Changes

Cohort / File(s)	Summary
Configuration config/openshift-customizations.yaml	Added a new releases.compliance-operator.jobs mapping with many boolean job entries (upstream ComplianceAsCode and downstream compliance-operator). No existing entries were removed or modified.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 6 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Title check	⚠️ Warning	The PR title 'Add a dashboard for Compliance Operator' is vague and does not match the actual changes, which only add compliance-operator job configuration entries to config/openshift-customizations.yaml.	Revise the title to accurately reflect the changes, such as 'Add compliance-operator jobs to OpenShift customizations' or 'Configure compliance-operator periodic jobs'.

✅ Passed checks (6 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Go Error Handling	✅ Passed	Go error handling check is not applicable; PR only modifies config/openshift-customizations.yaml YAML file with no Go code changes.
Sql Injection Prevention	✅ Passed	The PR modifies only config/openshift-customizations.yaml, a configuration file with static job mappings without SQL code or database operations.
Excessive Css In React Should Use Styles	✅ Passed	PR modifies only YAML configuration file with no React components containing inline styles.
Single Responsibility And Clear Naming	✅ Passed	The pull request successfully adheres to the Single Responsibility and Clear Naming principles. The new compliance-operator entry uses a clear, specific name that directly communicates its purpose—distinct from generic terms like Manager or Handler.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: xiaojiey
Once this PR has been reviewed and has the lgtm label, please assign dgoodwin for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@config/openshift-customizations.yaml`:
- Around line 35-103: The job name string
periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-privmaryv6-f60-compliance
contains a typo ("privmaryv6") that breaks CI naming; update that token to
"primaryv6" (resulting name:
periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-primaryv6-f60-compliance)
so the entry conforms to OpenShift CI naming and will resolve correctly in Prow.

---

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 76de618 and 15c5c57.

📒 Files selected for processing (1)

• config/openshift-customizations.yaml

[coderabbitai]

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# 1) First, extract and display the actual compliance-operator jobs from the file
echo "=== Extracting jobs from config/openshift-customizations.yaml ==="
jobs=$(awk '
  /^  compliance-operator:/ {in_block=1; next}
  in_block && /^[^ ]/ {in_block=0}
  in_block && /^      periodic-ci-/ {
    line=$0
    sub(/: true[[:space:]]*$/, "", line)
    gsub(/^[[:space:]]+/, "", line)
    print line
  }
' config/openshift-customizations.yaml)

echo "Found jobs:"
echo "$jobs" | nl
echo ""

# 2) Check for the suspected typo
echo "=== Checking for 'privmaryv6' typo ==="
if echo "$jobs" | grep -i "privmaryv6"; then
  echo "FOUND: privmaryv6 typo detected"
else
  echo "No 'privmaryv6' found"
fi
echo ""

# 3) Query Prow prowjobs.js
echo "=== Fetching Prow jobs catalog ==="
curl -fsSL "https://prow.ci.openshift.org/prowjobs.js" -o /tmp/prowjobs.js 2>&1 | head -5

if [ -f /tmp/prowjobs.js ]; then
  echo "Prow catalog fetched successfully"
  
  # 4) Verify each job exists in Prow
  echo ""
  echo "=== Verifying job existence in Prow ==="
  missing=0
  found=0
  while IFS= read -r j; do
    if grep -q "\"name\":\"$j\"" /tmp/prowjobs.js; then
      found=$((found + 1))
    else
      echo "MISSING: $j"
      missing=$((missing + 1))
    fi
  done <<< "$jobs"
  
  echo ""
  echo "Summary: Found=$found, Missing=$missing"
else
  echo "ERROR: Could not fetch Prow jobs catalog"
fi

Repository: openshift/sippy

Length of output: 197

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# 1) Check file exists and show the actual compliance-operator section
echo "=== Checking file and compliance-operator section ==="
if [ ! -f config/openshift-customizations.yaml ]; then
  echo "ERROR: config/openshift-customizations.yaml not found"
  exit 1
fi

# Show lines around compliance-operator to understand structure
echo "File content around compliance-operator:"
grep -n "compliance-operator" config/openshift-customizations.yaml | head -5
echo ""

# 2) Extract jobs using simpler grep + sed approach
echo "=== Extracting jobs ==="
jobs=$(grep "periodic-ci-" config/openshift-customizations.yaml | sed 's/.*periodic-ci-/periodic-ci-/' | sed 's/: true.*//')
echo "Total jobs extracted:"
echo "$jobs" | wc -l
echo ""

# 3) Show first few jobs
echo "First 10 jobs:"
echo "$jobs" | head -10
echo ""

# 4) Check for suspected typo
echo "=== Checking for 'privmaryv6' typo ==="
if echo "$jobs" | grep -i "privmaryv6"; then
  echo "FOUND: privmaryv6 typo detected"
else
  echo "No 'privmaryv6' found in job names"
fi
echo ""

# 5) Fetch Prow catalog
echo "=== Fetching Prow jobs catalog ==="
timeout 30 curl -fsSL "https://prow.ci.openshift.org/prowjobs.js" -o /tmp/prowjobs.js || {
  echo "Warning: Could not fetch from prow.ci.openshift.org, trying alternative source"
  timeout 30 curl -fsSL "https://prow.ci.openshift.org/api/prowjobs" -o /tmp/prowjobs.json || true
}

if [ -f /tmp/prowjobs.js ]; then
  echo "Prow catalog (prowjobs.js) fetched, size: $(wc -c < /tmp/prowjobs.js) bytes"
  echo "Sample of catalog:"
  head -c 500 /tmp/prowjobs.js
  echo ""
elif [ -f /tmp/prowjobs.json ]; then
  echo "Prow catalog (API JSON) fetched, size: $(wc -c < /tmp/prowjobs.json) bytes"
else
  echo "ERROR: Could not fetch Prow catalog from either endpoint"
fi

Repository: openshift/sippy

Length of output: 1373

---

🌐 Web query:

openshift vsphere dualstack ipv6 periodic test job name compliance

💡 Result:

In OpenShift CI/Prow, periodic job names are expected to follow the standard pattern:

periodic-ci-<org>-<repo>-<branch>-<test-context>

You can see this pattern in existing vSphere periodics like periodic-ci-openshift-release-master-nightly-4.14-e2e-vsphere-ovn-csi / ...-e2e-vsphere-ovn-serial / ...-e2e-vsphere-ovn-upi. [1]

For the vSphere OVN dual-stack lane, there is already a corresponding PR job named:

pull-ci-openshift-cluster-network-operator-master-e2e-vsphere-ovn-dualstack [2]

So, a compliant periodic name for the same lane would be:

periodic-ci-openshift-cluster-network-operator-master-e2e-vsphere-ovn-dualstack [2]

If you specifically need to distinguish IPv6-only vs dual-stack, OpenShift CI convention is to encode that in the trailing context (e.g., ...-ipv6 vs ...-dualstack), similar to other existing periodics (for non-vSphere platforms) that use ...-ovn-ipv6 and ...-ovn-dualstack. [3]

Sources:
[1] [4.14 nightly payload test list showing vSphere periodic job names] (amd64.ocp.releases.ci.openshift.org)
[2] [Existing vSphere OVN dual-stack CI job name (PR job)] (prow.ci.openshift.org)
[3] [Example periodic naming for ipv6 vs dualstack in payload tests] (openshift-release.apps.ci.l2s4.p1.openshiftapps.com)

Citations:

• 1: https://amd64.ocp.releases.ci.openshift.org/releasestream/4.14.0-0.nightly/release/4.14.0-0.nightly-2025-06-27-165823?utm_source=openai
• 2: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-network-operator/2556/pull-ci-openshift-cluster-network-operator-master-e2e-vsphere-ovn-dualstack/1902288597613023232?utm_source=openai
• 3: https://openshift-release.apps.ci.l2s4.p1.openshiftapps.com/releasestream/4.22.0-0.nightly/release/4.22.0-0.nightly-2026-02-16-211851?utm_source=openai

---

Fix malformed job name on line 66: privmaryv6 is a typo and will cause silent data loss.

Line 66 contains periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-privmaryv6-f60-compliance, where dualstack-privmaryv6 violates OpenShift CI naming conventions. The privmaryv6 token is malformed (likely a typo for primaryv6 or should be removed entirely). This job will not resolve in Prow, and the compliance dashboard will silently show no data for the 4.15 vSphere dual-stack compliance variant.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@config/openshift-customizations.yaml` around lines 35 - 103, The job name
string
periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-privmaryv6-f60-compliance
contains a typo ("privmaryv6") that breaks CI naming; update that token to
"primaryv6" (resulting name:
periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-primaryv6-f60-compliance)
so the entry conforms to OpenShift CI naming and will resolve correctly in Prow.

[coderabbitai]

♻️ Duplicate comments (1)

config/openshift-customizations.yaml (1)

66-67: ⚠️ Potential issue | 🔴 Critical

Fix malformed job token privmaryv6 to avoid missing dashboard data.

Line 66 and Line 67 still use dualstack-privmaryv6, which is a typo and will prevent correct job resolution.

Suggested fix

-      periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-privmaryv6-f60-compliance: true
-      periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-privmaryv6-f60-co-destructive: true
+      periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-primaryv6-f60-compliance: true
+      periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-primaryv6-f60-co-destructive: true

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@config/openshift-customizations.yaml` around lines 66 - 67, The two job keys
named
periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-privmaryv6-f60-compliance
and
periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-privmaryv6-f60-co-destructive
contain a typo ("privmaryv6"); update both keys to use "primaryv6"
(dualstack-primaryv6) so the job names resolve correctly and dashboard data is
not missed.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@config/openshift-customizations.yaml`:
- Around line 66-67: The two job keys named
periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-privmaryv6-f60-compliance
and
periodic-ci-openshift-openshift-tests-private-release-4.15-amd64-nightly-vsphere-ipi-ovn-dualstack-privmaryv6-f60-co-destructive
contain a typo ("privmaryv6"); update both keys to use "primaryv6"
(dualstack-primaryv6) so the job names resolve correctly and dashboard data is
not missed.

---

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 15c5c57 and 081fbef.

📒 Files selected for processing (1)

• config/openshift-customizations.yaml

[openshift-ci-robot]

Scheduling required tests:
/test e2e

[openshift-ci]

@xiaojiey: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.