Skip to content

🤖 Update Benchmark Results#5969

Merged
thomhurst merged 1 commit into
mainfrom
automated-benchmarks-update
May 20, 2026
Merged

🤖 Update Benchmark Results#5969
thomhurst merged 1 commit into
mainfrom
automated-benchmarks-update

Conversation

@thomhurst
Copy link
Copy Markdown
Owner

Automated Benchmark Update

This PR updates the benchmark documentation with the latest results from the Speed Comparison workflow.

Benchmarks Produced

Individual benchmark artifacts are available for download:

  • benchmark-DataDrivenTests
  • benchmark-AsyncTests
  • benchmark-ScaleTests
  • benchmark-MatrixTests
  • benchmark-MassiveParallelTests
  • benchmark-SetupTeardownTests
  • benchmark-BuildTime
  • benchmark-summary (aggregated overview)

Runtime Benchmarks

  • AsyncTests
  • DataDrivenTests
  • MassiveParallelTests
  • MatrixTests
  • ScaleTests
  • SetupTeardownTests

Build Benchmarks

  • BuildTime
  • 2026-05-20
  • Ubuntu Latest • .NET SDK 10.0.300

Changes

  • Updated benchmark data in docs/static/benchmarks/latest.json
  • Updated historical trends in docs/static/benchmarks/historical.json
  • Regenerated benchmark documentation in docs/docs/benchmarks/index.md
  • Updated benchmark summary in docs/static/benchmarks/summary.json

Workflow Run

  • Run ID: 26134500691
  • Triggered: schedule
  • Date:

🤖 This PR was automatically created and will be merged automatically once CI checks pass.

@thomhurst thomhurst added automated benchmarks documentation Improvements or additions to documentation ignore-for-release labels May 20, 2026
@thomhurst thomhurst merged commit c33d940 into main May 20, 2026
12 of 13 checks passed
@thomhurst thomhurst deleted the automated-benchmarks-update branch May 20, 2026 00:59
@codacy-production
Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 20, 2026

Code Review

This is an automated benchmark documentation update — no source-code changes. The diff is clean and internally consistent. A few observations worth noting for future context:

Environment shift affects all numbers

The benchmark host changed from x86-64-v3 → x86-64-v4 (a higher AVX-512 instruction-set tier). This means the new numbers are not directly comparable to previous runs in historical.json — a side-by-side trend analysis will silently mix two different hardware profiles. Consider adding a hostCpuFeature field (or similar) to each entry in historical.json so consumers can group by environment rather than mixing them.

Silent data-point removal in historical.json

The oldest entry (2026-02-22) was dropped without any recorded reason. If this is a rolling window, documenting the retention policy in the workflow that generates the file would make future removals self-explanatory and auditable.

NUnit version bump

NUnit 4.6.0 → 4.6.1 across all benchmarks — good, keeps the competitor baseline current.

Performance summary

TUnit results are mixed but generally positive: SetupTeardownTests improved ~9%, ScaleTests ~4%, BuildTime ~7%, with small regressions in AsyncTests (+0.7%) and DataDrivenTests (+6.3%). The environment change makes it hard to attribute these to code changes vs. CPU capability differences.

Overall the automation is working well. The main long-term improvement would be tagging historical entries with the host environment tier so the data stays comparable across machine upgrades.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated benchmarks documentation Improvements or additions to documentation ignore-for-release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant