Skip to content

conformance: add Hook in ConformanceTestSuite #3786

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 27, 2025
Merged

conformance: add Hook in ConformanceTestSuite #3786

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e675922
conformance: add Hook in ConformanceTestSuite
zirain May 11, 2025
a4456ae
update comment
zirain May 25, 2025
80c24e6
add Hook in ConformanceOptions
zirain May 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions conformance/utils/suite/suite.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ type ConformanceTestSuite struct {
SkipTests sets.Set[string]
SkipProvisionalTests bool
RunTest string
Hook func(t *testing.T, test ConformanceTest, suite *ConformanceTestSuite)
ManifestFS []fs.FS
UsableNetworkAddresses []v1beta1.GatewaySpecAddress
UnusableNetworkAddresses []v1beta1.GatewaySpecAddress
Expand Down Expand Up @@ -121,7 +122,7 @@ type ConformanceTestSuite struct {
lock sync.RWMutex
}

// Options can be used to initialize a ConformanceTestSuite.
// ConformanceOptions can be used to initialize a ConformanceTestSuite.
type ConformanceOptions struct {
Client client.Client
ClientOptions client.Options
Expand Down Expand Up @@ -152,7 +153,8 @@ type ConformanceOptions struct {
SkipProvisionalTests bool
// RunTest is a single test to run, mostly for development/debugging convenience.
RunTest string

// Hook is an optional function that can be used to run custom logic after each test at suite level.
Hook func(t *testing.T, test ConformanceTest, suite *ConformanceTestSuite)
ManifestFS []fs.FS

// UsableNetworkAddresses is an optional pool of usable addresses for
Expand Down Expand Up @@ -268,6 +270,7 @@ func NewConformanceTestSuite(options ConformanceOptions) (*ConformanceTestSuite,
mode: mode,
apiVersion: apiVersion,
apiChannel: apiChannel,
Hook: options.Hook,
}

for _, conformanceProfileName := range options.ConformanceProfiles.UnsortedList() {
Expand Down Expand Up @@ -470,6 +473,13 @@ func (suite *ConformanceTestSuite) Run(t *testing.T, tests []ConformanceTest) er
if res == testSucceeded || res == testFailed {
sleepForTestIsolation = true
}

// call the hook function if it was provided,
// this's useful for running custom logic after each test at suite level,
// such as collecting current state of the cluster for debugging.
if suite.Hook != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: is it only useful on the suite level, meaning it will run the same hook for each test? or there is value in doing it on a more granular level like tests/profile/supportedFeature etc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it's for suite level.
For specified test, you can use t.Cleanup()

suite.Hook(t, test, suite)
}
}

// now that the tests have completed, mark the test suite as not running
Expand Down