Skip to content

Commit 512b71c

Browse files
thomhurstthomhurst
authored
chore: update benchmark results (#4934)
1 parent 065bbca commit 512b71c

18 files changed

Lines changed: 392 additions & 392 deletions

‎docs/docs/benchmarks/AsyncTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 2
77
# AsyncTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-18** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-19** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.103
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2026-02-18** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.15.7 | 544.2 ms | 543.7 ms | 11.45 ms |
20-
| NUnit | 4.4.0 | 713.0 ms | 712.6 ms | 7.57 ms |
21-
| MSTest | 4.1.0 | 649.0 ms | 649.5 ms | 6.90 ms |
22-
| xUnit3 | 3.2.2 | 782.6 ms | 783.3 ms | 10.62 ms |
23-
| **TUnit (AOT)** | 1.15.7 | 122.1 ms | 122.0 ms | 0.40 ms |
19+
| **TUnit** | 1.16.4 | 536.5 ms | 535.2 ms | 5.40 ms |
20+
| NUnit | 4.5.0 | 705.3 ms | 703.4 ms | 7.66 ms |
21+
| MSTest | 4.1.0 | 641.0 ms | 636.7 ms | 10.04 ms |
22+
| xUnit3 | 3.2.2 | 773.6 ms | 769.8 ms | 10.41 ms |
23+
| **TUnit (AOT)** | 1.16.4 | 123.6 ms | 123.6 ms | 0.44 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2026-02-18** from the latest CI
5858
xychart-beta
5959
title "AsyncTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 940
62-
bar [544.2, 713, 649, 782.6, 122.1]
61+
y-axis "Time (ms)" 0 --> 929
62+
bar [536.5, 705.3, 641, 773.6, 123.6]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2026-02-18T00:37:45.108Z*
75+
*Last generated: 2026-02-19T01:56:15.278Z*

‎docs/docs/benchmarks/BuildTime.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 8
77
# Build Performance Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-18** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-19** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.103
1313
:::
@@ -18,10 +18,10 @@ Compilation time comparison across frameworks:
1818

1919
| Framework | Version | Mean | Median | StdDev |
2020
|-----------|---------|------|--------|--------|
21-
| **TUnit** | 1.15.7 | 2.066 s | 2.071 s | 0.0336 s |
22-
| Build_NUnit | 4.4.0 | 1.675 s | 1.677 s | 0.0235 s |
23-
| Build_MSTest | 4.1.0 | 1.747 s | 1.747 s | 0.0184 s |
24-
| Build_xUnit3 | 3.2.2 | 1.651 s | 1.640 s | 0.0228 s |
21+
| **TUnit** | 1.16.4 | 2.213 s | 2.208 s | 0.0546 s |
22+
| Build_NUnit | 4.5.0 | 1.824 s | 1.823 s | 0.0291 s |
23+
| Build_MSTest | 4.1.0 | 1.898 s | 1.896 s | 0.0230 s |
24+
| Build_xUnit3 | 3.2.2 | 1.808 s | 1.811 s | 0.0139 s |
2525

2626
## 📈 Visual Comparison
2727

@@ -60,7 +60,7 @@ xychart-beta
6060
title "Build Time Comparison"
6161
x-axis ["Build_TUnit", "Build_NUnit", "Build_MSTest", "Build_xUnit3"]
6262
y-axis "Time (s)" 0 --> 3
63-
bar [2.066, 1.675, 1.747, 1.651]
63+
bar [2.213, 1.824, 1.898, 1.808]
6464
```
6565

6666
---
@@ -69,4 +69,4 @@ xychart-beta
6969
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7070
:::
7171

72-
*Last generated: 2026-02-18T00:37:45.110Z*
72+
*Last generated: 2026-02-19T01:56:15.280Z*

‎docs/docs/benchmarks/DataDrivenTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 3
77
# DataDrivenTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-18** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-19** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.103
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2026-02-18** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.15.7 | 435.95 ms | 434.78 ms | 4.660 ms |
20-
| NUnit | 4.4.0 | 640.77 ms | 636.95 ms | 11.657 ms |
21-
| MSTest | 4.1.0 | 637.84 ms | 636.45 ms | 9.710 ms |
22-
| xUnit3 | 3.2.2 | 652.36 ms | 650.38 ms | 9.717 ms |
23-
| **TUnit (AOT)** | 1.15.7 | 23.74 ms | 23.81 ms | 1.072 ms |
19+
| **TUnit** | 1.16.4 | 476.93 ms | 476.23 ms | 3.201 ms |
20+
| NUnit | 4.5.0 | 607.45 ms | 613.30 ms | 27.836 ms |
21+
| MSTest | 4.1.0 | 626.12 ms | 636.08 ms | 31.249 ms |
22+
| xUnit3 | 3.2.2 | 653.78 ms | 651.57 ms | 18.795 ms |
23+
| **TUnit (AOT)** | 1.16.4 | 25.52 ms | 25.59 ms | 0.650 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2026-02-18** from the latest CI
5858
xychart-beta
5959
title "DataDrivenTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 783
62-
bar [435.95, 640.77, 637.84, 652.36, 23.74]
61+
y-axis "Time (ms)" 0 --> 785
62+
bar [476.93, 607.45, 626.12, 653.78, 25.52]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2026-02-18T00:37:45.108Z*
75+
*Last generated: 2026-02-19T01:56:15.278Z*

‎docs/docs/benchmarks/MassiveParallelTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 4
77
# MassiveParallelTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-18** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-19** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.103
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2026-02-18** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.15.7 | 655.3 ms | 654.0 ms | 5.35 ms |
20-
| NUnit | 4.4.0 | 1,182.5 ms | 1,182.2 ms | 5.21 ms |
21-
| MSTest | 4.1.0 | 2,933.9 ms | 2,933.8 ms | 4.67 ms |
22-
| xUnit3 | 3.2.2 | 3,057.6 ms | 3,058.2 ms | 4.37 ms |
23-
| **TUnit (AOT)** | 1.15.7 | 228.9 ms | 228.7 ms | 0.64 ms |
19+
| **TUnit** | 1.16.4 | 657.3 ms | 656.2 ms | 5.47 ms |
20+
| NUnit | 4.5.0 | 1,200.9 ms | 1,195.3 ms | 12.41 ms |
21+
| MSTest | 4.1.0 | 2,945.5 ms | 2,944.4 ms | 5.81 ms |
22+
| xUnit3 | 3.2.2 | 3,076.3 ms | 3,076.5 ms | 10.28 ms |
23+
| **TUnit (AOT)** | 1.16.4 | 227.7 ms | 227.7 ms | 0.58 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2026-02-18** from the latest CI
5858
xychart-beta
5959
title "MassiveParallelTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 3670
62-
bar [655.3, 1182.5, 2933.9, 3057.6, 228.9]
61+
y-axis "Time (ms)" 0 --> 3692
62+
bar [657.3, 1200.9, 2945.5, 3076.3, 227.7]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2026-02-18T00:37:45.108Z*
75+
*Last generated: 2026-02-19T01:56:15.278Z*

‎docs/docs/benchmarks/MatrixTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 5
77
# MatrixTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-18** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-19** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.103
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2026-02-18** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.15.7 | 579.7 ms | 578.0 ms | 6.69 ms |
20-
| NUnit | 4.4.0 | 1,581.0 ms | 1,581.2 ms | 5.83 ms |
21-
| MSTest | 4.1.0 | 1,509.7 ms | 1,511.1 ms | 5.24 ms |
22-
| xUnit3 | 3.2.2 | 1,637.4 ms | 1,639.7 ms | 5.65 ms |
23-
| **TUnit (AOT)** | 1.15.7 | 127.1 ms | 127.2 ms | 0.32 ms |
19+
| **TUnit** | 1.16.4 | 575.8 ms | 574.3 ms | 5.81 ms |
20+
| NUnit | 4.5.0 | 1,591.7 ms | 1,593.9 ms | 7.60 ms |
21+
| MSTest | 4.1.0 | 1,508.2 ms | 1,506.1 ms | 7.67 ms |
22+
| xUnit3 | 3.2.2 | 1,643.8 ms | 1,643.5 ms | 6.65 ms |
23+
| **TUnit (AOT)** | 1.16.4 | 127.2 ms | 127.1 ms | 0.41 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2026-02-18** from the latest CI
5858
xychart-beta
5959
title "MatrixTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1965
62-
bar [579.7, 1581, 1509.7, 1637.4, 127.1]
61+
y-axis "Time (ms)" 0 --> 1973
62+
bar [575.8, 1591.7, 1508.2, 1643.8, 127.2]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2026-02-18T00:37:45.109Z*
75+
*Last generated: 2026-02-19T01:56:15.279Z*

‎docs/docs/benchmarks/ScaleTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 6
77
# ScaleTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-18** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-19** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.103
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2026-02-18** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.15.7 | 492.70 ms | 492.28 ms | 4.681 ms |
20-
| NUnit | 4.4.0 | 627.91 ms | 627.70 ms | 14.071 ms |
21-
| MSTest | 4.1.0 | 607.82 ms | 606.35 ms | 25.642 ms |
22-
| xUnit3 | 3.2.2 | 658.98 ms | 660.13 ms | 12.793 ms |
23-
| **TUnit (AOT)** | 1.15.7 | 35.09 ms | 35.25 ms | 2.916 ms |
19+
| **TUnit** | 1.16.4 | 476.89 ms | 477.05 ms | 2.478 ms |
20+
| NUnit | 4.5.0 | 597.73 ms | 596.41 ms | 6.112 ms |
21+
| MSTest | 4.1.0 | 551.41 ms | 550.35 ms | 18.242 ms |
22+
| xUnit3 | 3.2.2 | 616.73 ms | 614.88 ms | 6.841 ms |
23+
| **TUnit (AOT)** | 1.16.4 | 35.91 ms | 35.73 ms | 2.978 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2026-02-18** from the latest CI
5858
xychart-beta
5959
title "ScaleTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 791
62-
bar [492.7, 627.91, 607.82, 658.98, 35.09]
61+
y-axis "Time (ms)" 0 --> 741
62+
bar [476.89, 597.73, 551.41, 616.73, 35.91]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2026-02-18T00:37:45.109Z*
75+
*Last generated: 2026-02-19T01:56:15.279Z*

‎docs/docs/benchmarks/SetupTeardownTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 7
77
# SetupTeardownTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2026-02-18** from the latest CI run.
10+
This benchmark was automatically generated on **2026-02-19** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.103
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2026-02-18** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.15.7 | 561.7 ms | 560.1 ms | 5.02 ms |
20-
| NUnit | 4.4.0 | 1,147.6 ms | 1,145.7 ms | 12.54 ms |
21-
| MSTest | 4.1.0 | 1,091.3 ms | 1,090.0 ms | 7.80 ms |
22-
| xUnit3 | 3.2.2 | 1,215.1 ms | 1,212.3 ms | 7.93 ms |
23-
| **TUnit (AOT)** | 1.15.7 | NA | NA | NA |
19+
| **TUnit** | 1.16.4 | 581.8 ms | 576.7 ms | 9.42 ms |
20+
| NUnit | 4.5.0 | 1,185.3 ms | 1,185.2 ms | 6.33 ms |
21+
| MSTest | 4.1.0 | 1,118.8 ms | 1,120.0 ms | 6.26 ms |
22+
| xUnit3 | 3.2.2 | 1,243.8 ms | 1,244.4 ms | 7.27 ms |
23+
| **TUnit (AOT)** | 1.16.4 | NA | NA | NA |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2026-02-18** from the latest CI
5858
xychart-beta
5959
title "SetupTeardownTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1459
62-
bar [561.7, 1147.6, 1091.3, 1215.1, 0]
61+
y-axis "Time (ms)" 0 --> 1493
62+
bar [581.8, 1185.3, 1118.8, 1243.8, 0]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2026-02-18T00:37:45.109Z*
75+
*Last generated: 2026-02-19T01:56:15.279Z*

‎docs/docs/benchmarks/index.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 1
77
# Performance Benchmarks
88

99
:::info Last Updated
10-
These benchmarks were automatically generated on **2026-02-18** from the latest CI run.
10+
These benchmarks were automatically generated on **2026-02-19** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.103
1313
:::
@@ -37,9 +37,9 @@ These benchmarks compare TUnit against the most popular .NET testing frameworks:
3737

3838
| Framework | Version Tested |
3939
|-----------|----------------|
40-
| **TUnit** | 1.15.7 |
40+
| **TUnit** | 1.16.4 |
4141
| **xUnit v3** | 3.2.2 |
42-
| **NUnit** | 4.4.0 |
42+
| **NUnit** | 4.5.0 |
4343
| **MSTest** | 4.1.0 |
4444

4545
### Test Scenarios
@@ -56,7 +56,7 @@ The benchmarks measure real-world testing patterns:
5656
### Environment
5757

5858
- **OS**: Ubuntu Latest (GitHub Actions)
59-
- **Runtime**: .NET 10.0.3 (10.0.3, 10.0.326.7603), X64 RyuJIT x86-64-v4
59+
- **Runtime**: .NET 10.0.3 (10.0.3, 10.0.326.7603), X64 RyuJIT x86-64-v3
6060
- **SDK**: .NET SDK 10.0.103
6161
- **Hardware**: GitHub Actions Standard Runner (Ubuntu)
6262
- **Tool**: BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.3 LTS (Noble Numbat)
@@ -80,4 +80,4 @@ These benchmarks run automatically daily via [GitHub Actions](https://github.com
8080
Each benchmark runs multiple iterations with statistical analysis to ensure accuracy. Results may vary based on hardware and test characteristics.
8181
:::
8282

83-
*Last generated: 2026-02-18T00:37:45.110Z*
83+
*Last generated: 2026-02-19T01:56:15.280Z*

0 commit comments

Comments
 (0)