Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[skip-ci] TMT: system tests #5885

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[skip-ci] TMT: system tests #5885

wants to merge 1 commit into from

Conversation

lsm5
Copy link
Member

@lsm5 lsm5 commented Dec 20, 2024 •
edited
Loading

This commit adds TMT jobs for system tests triggered via Packit for all active Fedora and CentOS Stream releases on x86_64.

TODO: enable aarch64 tests.

Official Fedora and CentOS Stream spec and gating test configs can be synced from upstream by Packit, effectively upstreaming almost all mainteenance.

What type of PR is this?

/kind other

What this PR does / why we need it:

Runs system tests using TMT. This test setup can be reused for official Fedora and CentOS-Stream packages as well as reused for upstream reverse dependency testing in c/common.

How to verify it

TMT tests in CI list.

Which issue(s) this PR fixes:

Special notes for your reviewer:

None

Does this PR introduce a user-facing change?

None

@packit-as-a-service Packit-as-a-Service
Copy link

Failed to load packit config file:

Cannot parse package config. ValidationError({'jobs[4].packages': 'Undefined package(s) referenced: podman-fedora.', 'jobs[5].packages': 'Undefined package(s) referenced: podman-fedora.', 'jobs[6].packages': 'Undefined package(s) referenced: podman-centos.', 'jobs[7].packages': 'Undefined package(s) referenced: podman-centos.'})

For more info, please check out the documentation or contact the Packit team. You can also use our CLI command validate-config or our pre-commit hooks for validation of the configuration.

2 similar comments
@packit-as-a-service Packit-as-a-Service
Copy link

Failed to load packit config file:

Cannot parse package config. ValidationError({'jobs[4].packages': 'Undefined package(s) referenced: podman-fedora.', 'jobs[5].packages': 'Undefined package(s) referenced: podman-fedora.', 'jobs[6].packages': 'Undefined package(s) referenced: podman-centos.', 'jobs[7].packages': 'Undefined package(s) referenced: podman-centos.'})

For more info, please check out the documentation or contact the Packit team. You can also use our CLI command validate-config or our pre-commit hooks for validation of the configuration.

@packit-as-a-service Packit-as-a-Service
Copy link

Failed to load packit config file:

Cannot parse package config. ValidationError({'jobs[4].packages': 'Undefined package(s) referenced: podman-fedora.', 'jobs[5].packages': 'Undefined package(s) referenced: podman-fedora.', 'jobs[6].packages': 'Undefined package(s) referenced: podman-centos.', 'jobs[7].packages': 'Undefined package(s) referenced: podman-centos.'})

For more info, please check out the documentation or contact the Packit team. You can also use our CLI command validate-config or our pre-commit hooks for validation of the configuration.

@lsm5 lsm5 force-pushed the tmt-system-tests branch 7 times, most recently from bf833e8 to e9a0088 Compare December 24, 2024 11:02
@lsm5
Copy link
Member Author

lsm5 commented Dec 24, 2024

/packit retest

@lsm5
Copy link
Member Author

lsm5 commented Dec 24, 2024

/packit test

@lsm5 lsm5 force-pushed the tmt-system-tests branch 3 times, most recently from 1a74573 to 7b52fe4 Compare December 25, 2024 10:05
@lsm5 lsm5 force-pushed the tmt-system-tests branch 3 times, most recently from 2c7daef to 429c88f Compare January 20, 2025 10:09
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 1, 2025
@lsm5 lsm5 force-pushed the tmt-system-tests branch from 429c88f to 2a2c4bf Compare February 4, 2025 12:52
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 4, 2025
@lsm5 lsm5 force-pushed the tmt-system-tests branch from 2a2c4bf to a2503ff Compare February 4, 2025 13:00
@lsm5
Copy link
Member Author

lsm5 commented Feb 4, 2025 •
edited
Loading

@flouthoc @nalind we're not testing on aarch64 yet are we?

some of the testing-farm-*-aarch64 tests added in this PR are failing on linux/arm64 vs linux/amd64 and some cases are warning on it and failing like so:

# # /usr/bin/buildah commit --signature-policy /usr/share/buildah/test/system/policy.json time="2025-02-04T13:50:22Z" level=warning msg="image platform (linux/amd64) does not match the expected platform (linux/arm64)" testdigest_v2s2-working-container new-image
# Error: too many arguments specified
# [ rc=125 (** EXPECTED 0 **) ]

@nalind
Copy link
Member

nalind commented Feb 4, 2025

@flouthoc @nalind we're not testing on aarch64 yet are we?

some of the testing-farm-*-aarch64 tests added in this PR are failing on linux/arm64 vs linux/amd64 and some cases are warning on it and failing like so:

# # /usr/bin/buildah commit --signature-policy /usr/share/buildah/test/system/policy.json time="2025-02-04T13:50:22Z" level=warning msg="image platform (linux/amd64) does not match the expected platform (linux/arm64)" testdigest_v2s2-working-container new-image
# Error: too many arguments specified
# [ rc=125 (** EXPECTED 0 **) ]

We're not testing on aarch64 - this looks like a previous line in the test that wanted to capture the ID of a newly-created working container captured the whole line time="2025-02-04T13:50:22Z" level=warning msg="image platform (linux/amd64) does not match the expected platform (linux/arm64)" as its name.

@nalind
Copy link
Member

nalind commented Feb 4, 2025

Oh, I guess the testing farm is testing natively on non-amd64. Nice!

@lsm5 lsm5 force-pushed the tmt-system-tests branch from a2503ff to 52c82c8 Compare February 5, 2025 10:50
@packit-as-a-service Packit-as-a-Service
Copy link

Ephemeral COPR build failed. @containers/packit-build please check.

@lsm5
Copy link
Member Author

lsm5 commented Feb 6, 2025

Oh, I guess the testing farm is testing natively on non-amd64. Nice!

@nalind @flouthoc what do you suggest? Should I only enable testing-farm for x86_64 for now? I'd be cool with that. Official fedora tests only x86_64 for now, and IIUC buildah ain't a requirement for RHIVOS which does require aarch64.

@lsm5 lsm5 force-pushed the tmt-system-tests branch 2 times, most recently from e0138e8 to 1261789 Compare February 10, 2025 13:06
@lsm5 lsm5 marked this pull request as ready for review February 10, 2025 14:04
@lsm5
Copy link
Member Author

lsm5 commented Feb 10, 2025

Opening up for review. I've disabled aarch64 test jobs for now.

@lsm5 lsm5 force-pushed the tmt-system-tests branch 2 times, most recently from 330996f to 78d4a63 Compare February 14, 2025 13:49
@TomSweeneyRedHat
Copy link
Member

LGTM
but certainly not my bailiwick.

@lsm5 lsm5 force-pushed the tmt-system-tests branch 2 times, most recently from dc36b3e to 6224a92 Compare February 19, 2025 10:29
@lsm5
Copy link
Member Author

lsm5 commented Feb 19, 2025

@nalind @flouthoc @TomSweeneyRedHat let's get this merged unless you got further concerns.

flouthoc
flouthoc approved these changes Feb 19, 2025
View reviewed changes
Copy link
Collaborator

@flouthoc flouthoc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@TomSweeneyRedHat
Copy link
Member

@lsm5 looks like a rebase might be needed.

@lsm5
Copy link
Member Author

lsm5 commented Feb 20, 2025

@lsm5 looks like a rebase might be needed.

Done

@lsm5
Copy link
Member Author

lsm5 commented Feb 20, 2025

@containers/buildah-maintainers PTAL.

flouthoc
flouthoc approved these changes Feb 20, 2025
View reviewed changes
Copy link
Collaborator

@flouthoc flouthoc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
/lgtm
/approve

@openshift-ci openshift-ci bot added the lgtm label Feb 20, 2025
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Feb 20, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: flouthoc, lsm5

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@flouthoc
Copy link
Collaborator

/hold

@flouthoc
Copy link
Collaborator

@nalind PTAL

@lsm5
[skip-ci] TMT: system tests
9158b65 
This commit adds TMT jobs for system tests triggered via Packit for
all active Fedora and CentOS Stream releases on x86_64.

TODO: enable aarch64 tests.

Official Fedora and CentOS Stream spec and gating test configs can be
synced from upstream by Packit, effectively upstreaming almost all
mainteenance.

Signed-off-by: Lokesh Mandvekar <[email protected]>
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Feb 20, 2025

New changes are detected. LGTM label has been removed.

@openshift-ci openshift-ci bot removed the lgtm label Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@flouthoc flouthoc flouthoc approved these changes

Assignees

@flouthoc flouthoc

Labels
approved do-not-merge/hold kind/other
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@lsm5 @nalind @TomSweeneyRedHat @flouthoc @openshift-merge-robot