Skip to content

Show individual test results in functional test logs#11917

Closed
brooke-hamilton wants to merge 1 commit into
mainfrom
brooke-hamilton/functional-test-display
Closed

Show individual test results in functional test logs#11917
brooke-hamilton wants to merge 1 commit into
mainfrom
brooke-hamilton/functional-test-display

Conversation

@brooke-hamilton
Copy link
Copy Markdown
Member

@brooke-hamilton brooke-hamilton commented May 15, 2026

Purpose

Show pass/fail status and elapsed time for each individual functional test in CI logs, instead of one consolidated line per package. This will help us find and optimize tests with lengthy execution times.

Changes

Adds --format testname --format-hide-empty-pkg to GOTESTSUM_OPTS in both functional test workflows:

  • .github/workflows/functional-test-cloud.yaml
  • .github/workflows/functional-test-noncloud.yaml

Before

gotestsum defaulted to pkgname format, producing one line per package:

✓  test/functional-portable/corerp/cloud/mechanics (56.015s)
✓  test/functional-portable/corerp/cloud/resources (19m42.189s)

Individual test names and timings were hidden.

After

testname format prints a line per test as it completes (with PASS/FAIL/SKIP and elapsed time). --format-hide-empty-pkg suppresses noise from packages that contain no tests.

::group::PASS test/functional-portable/samples/noncloud.Test_FirstApplicationSample/deploy_testdata/tutorial-environment.bicep (14.26s)
PASS test/functional-portable/samples/noncloud.Test_FirstApplicationSample/deploy_testdata/tutorial-environment.bicep (14.26s)
::group::PASS test/functional-portable/samples/noncloud.Test_FirstApplicationSample/deploy_../../../../samples/samples/demo/app.bicep (37.94s)
PASS test/functional-portable/samples/noncloud.Test_FirstApplicationSample/deploy_../../../../samples/samples/demo/app.bicep (37.94s)
::group::PASS test/functional-portable/samples/noncloud.Test_FirstApplicationSample (52.22s)
PASS test/functional-portable/samples/noncloud.Test_FirstApplicationSample (52.22s)
  PASS Package test/functional-portable/samples/noncloud (52.248s)

References

Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@radius-functional-tests
Copy link
Copy Markdown

radius-functional-tests Bot commented May 15, 2026

Radius functional test overview

🔍 Go to test action run

Click here to see the test run details
Name Value
Repository radius-project/radius
Commit ref de5e841
Unique ID funcb512fd9626
Image tag pr-funcb512fd9626
  • gotestsum 1.13.0
  • KinD: v0.29.0
  • Dapr: 1.14.4
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcb512fd9626
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcb512fd9626
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-funcb512fd9626
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-funcb512fd9626
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcb512fd9626
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

@brooke-hamilton brooke-hamilton marked this pull request as ready for review May 15, 2026 23:15
@brooke-hamilton brooke-hamilton requested review from a team as code owners May 15, 2026 23:15
Copilot AI review requested due to automatic review settings May 15, 2026 23:15
@brooke-hamilton brooke-hamilton requested a review from a team as a code owner May 15, 2026 23:15
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the functional test workflows so CI logs show per-test pass/fail status and elapsed time from gotestsum, making slow functional tests easier to identify.

Changes:

  • Adds --format testname to functional test GOTESTSUM_OPTS.
  • Adds --format-hide-empty-pkg to suppress empty package noise.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
.github/workflows/functional-test-cloud.yaml Updates cloud functional test gotestsum output formatting.
.github/workflows/functional-test-noncloud.yaml Updates non-cloud functional test gotestsum output formatting.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 15, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 51.71%. Comparing base (7766024) to head (de5e841).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #11917      +/-   ##
==========================================
- Coverage   51.72%   51.71%   -0.02%     
==========================================
  Files         726      726              
  Lines       45608    45608              
==========================================
- Hits        23593    23587       -6     
- Misses      19792    19795       +3     
- Partials     2223     2226       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions
Copy link
Copy Markdown

Unit Tests

    2 files  ±0    423 suites  ±0   7m 5s ⏱️ +4s
5 128 tests ±0  5 126 ✅ ±0  2 💤 ±0  0 ❌ ±0 
6 157 runs  ±0  6 155 ✅ ±0  2 💤 ±0  0 ❌ ±0 

Results for commit de5e841. ± Comparison against base commit 7766024.

@brooke-hamilton brooke-hamilton marked this pull request as draft May 16, 2026 00:36
@brooke-hamilton
Copy link
Copy Markdown
Member Author

Closing PR. This change caused the non-cloud tests to hang.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants