🤖 Update Mock Benchmark Results

[thomhurst]

Automated Mock Benchmark Update

This PR updates the mock benchmark documentation with the latest results from the Mock Benchmarks workflow.

Benchmarks Produced

Individual benchmark artifacts are available for download:

• mock-benchmark-MockCreation
• mock-benchmark-Setup
• mock-benchmark-Invocation
• mock-benchmark-Verification
• mock-benchmark-Callback
• mock-benchmark-CombinedWorkflow
• mock-benchmark-summary (aggregated overview)

Libraries Compared

Libraries are discovered dynamically from the benchmark output. See the generated summary.json for the full list.

Changes

• Updated mock benchmark data in docs/static/benchmarks/mocks/latest.json
• Regenerated mock benchmark documentation in docs/docs/benchmarks/mocks/
• Updated mock benchmark summary in docs/static/benchmarks/mocks/summary.json

Workflow Run

• Run ID: 26993139016
• Triggered: schedule

---

🤖 This PR was automatically created and will be merged automatically once CI checks pass.

[claude]

Automated Benchmark Update Review

This PR is a routine automated update to mock benchmark documentation — no production code changes. The automation mechanism looks correct: scheduled trigger, data regenerated from CI artifacts, markdown and JSON both updated consistently, timestamps advanced.

Data Quality Observation

A few benchmark values warrant attention, not as a blocker but as context for readers:

High error-to-mean ratios in Invocation benchmarks — some entries have error bars that are a large fraction of the mean. For example in the new data:

• Imposter: Mean 241.29 ns, Error 186.342 ns (≈77% of mean)
• NSubstitute: Mean 607.70 ns, Error 516.136 ns (≈85% of mean)

These wide error intervals make the absolute numbers less meaningful. This is a known challenge with micro-benchmarking on shared CI runners (CPU throttling, noisy neighbours), but readers comparing numbers day-to-day may draw incorrect conclusions.

Large Moq regressions in a single day — Moq's CombinedWorkflow result went from 312 μs to 406 μs (~30% slower), and Verification from 246 μs to 374 μs (~52% slower). No Moq code changed, so this is almost certainly CI noise. If these numbers end up in a public-facing comparison, they could unfairly misrepresent Moq's actual performance.

Suggestions (not blockers)

1. Consider adding a footnote or tooltip in the generated markdown for benchmarks where , flagging them as high-variance. This would prevent misleading day-to-day comparisons.
2. For the summary/overview page, using a rolling average (e.g. last 7 runs) rather than a single-run snapshot would smooth out CI noise and make the numbers more trustworthy.

Summary

The automation itself is well-structured and the PR is correct as generated. The above are suggestions for improving benchmark data quality over time, not issues with this specific update.

[claude]

Automated Benchmark Update Review

This PR is a routine automated update to mock benchmark documentation — no production code changes. The automation mechanism looks correct: scheduled trigger, data regenerated from CI artifacts, markdown and JSON both updated consistently, timestamps advanced.

Data Quality Observation

A few benchmark values warrant attention, not as a blocker but as context for readers:

High error-to-mean ratios in Invocation benchmarks — some entries have error bars that are a large fraction of the mean. For example in the new data:

• Imposter: Mean 241.29 ns, Error 186.342 ns (≈77% of mean)
• NSubstitute: Mean 607.70 ns, Error 516.136 ns (≈85% of mean)

These wide error intervals make the absolute numbers less meaningful. This is a known challenge with micro-benchmarking on shared CI runners (CPU throttling, noisy neighbours), but readers comparing numbers day-to-day may draw incorrect conclusions.

Large Moq regressions in a single day — Moq's CombinedWorkflow result went from 312 μs to 406 μs (~30% slower), and Verification from 246 μs to 374 μs (~52% slower). No Moq code changed, so this is almost certainly CI noise. If these numbers end up in a public-facing comparison, they could unfairly misrepresent Moq's actual performance.

Suggestions (not blockers)

1. Consider adding a footnote or tooltip in the generated markdown for benchmarks where Error / Mean > 20%, flagging them as high-variance. This would prevent misleading day-to-day comparisons.
2. For the summary/overview page, using a rolling average (e.g. last 7 runs) rather than a single-run snapshot would smooth out CI noise and make the numbers more trustworthy.

Summary

The automation itself is well-structured and the PR is correct as generated. The above are suggestions for improving benchmark data quality over time, not issues with this specific update.

[codacy-production]

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}