Skip to content

Only run asserts in debug mode #66

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: lh/anew-dev
Choose a base branch
from
Open

Conversation

LilithHafner
Copy link
Owner

This is motivated by the ease of performing the optimization, not by profile guided optimization which is a huge red flag.

Copy link

github-actions bot commented Feb 5, 2025

Benchmark Results

e04e15c... 4756eb9... e04e15c.../4756eb9021ba5b...
constructor n=100 σ=0.1 9.49 ± 3.2 μs 10.9 ± 1.3 μs 0.87
constructor n=100 σ=1.0 9.43 ± 0.89 μs 12.9 ± 2.9 μs 0.729
constructor n=100 σ=10.0 9.47 ± 0.46 μs 11.3 ± 1 μs 0.838
constructor n=100 σ=100.0 10.1 ± 3.4 μs 10.1 ± 2.9 μs 1.01
constructor n=1000 σ=0.1 0.0569 ± 0.0026 ms 0.0579 ± 0.0038 ms 0.983
constructor n=1000 σ=1.0 0.0591 ± 0.0028 ms 0.0593 ± 0.004 ms 0.996
constructor n=1000 σ=10.0 0.0647 ± 0.0018 ms 0.075 ± 0.0019 ms 0.863
constructor n=1000 σ=100.0 0.0727 ± 0.0019 ms 0.0792 ± 0.0026 ms 0.917
constructor n=10000 σ=0.1 0.58 ± 0.11 ms 0.61 ± 0.05 ms 0.951
constructor n=10000 σ=1.0 0.919 ± 0.41 ms 0.626 ± 0.029 ms 1.47
constructor n=10000 σ=10.0 1.02 ± 0.53 ms 0.648 ± 0.28 ms 1.57
constructor n=10000 σ=100.0 0.74 ± 0.1 ms 0.813 ± 0.057 ms 0.911
delete ∘ rand n=100 σ=0.1 5.59 ± 0.21 μs 5.79 ± 0.23 μs 0.965
delete ∘ rand n=100 σ=1.0 5.9 ± 0.21 μs 6.01 ± 0.23 μs 0.982
delete ∘ rand n=100 σ=10.0 7.18 ± 0.43 μs 7.19 ± 0.43 μs 0.999
delete ∘ rand n=100 σ=100.0 16.4 ± 0.86 μs 15.8 ± 0.86 μs 1.04
delete ∘ rand n=1000 σ=0.1 0.0557 ± 0.0012 ms 0.0575 ± 0.0012 ms 0.969
delete ∘ rand n=1000 σ=1.0 0.0604 ± 0.005 ms 0.0614 ± 0.0012 ms 0.984
delete ∘ rand n=1000 σ=10.0 0.0626 ± 0.001 ms 0.0635 ± 0.001 ms 0.986
delete ∘ rand n=1000 σ=100.0 0.0866 ± 0.0018 ms 0.0845 ± 0.0014 ms 1.02
delete ∘ rand n=10000 σ=0.1 0.62 ± 0.021 ms 0.611 ± 0.015 ms 1.01
delete ∘ rand n=10000 σ=1.0 0.646 ± 0.014 ms 0.654 ± 0.013 ms 0.986
delete ∘ rand n=10000 σ=10.0 0.651 ± 0.013 ms 0.66 ± 0.018 ms 0.987
delete ∘ rand n=10000 σ=100.0 0.666 ± 0.012 ms 0.671 ± 0.014 ms 0.993
empty constructor 3.61 ± 1.3 μs 4.25 ± 1.1 μs 0.849
intermixed_h n=100 σ=0.1 14.2 ± 1.7 μs 16.1 ± 2.5 μs 0.88
intermixed_h n=100 σ=1.0 14.6 ± 1.8 μs 15.4 ± 1.9 μs 0.952
intermixed_h n=100 σ=10.0 18 ± 3.8 μs 16.9 ± 3.7 μs 1.07
intermixed_h n=100 σ=100.0 17.1 ± 2.3 μs 18.6 ± 2.8 μs 0.919
intermixed_h n=1000 σ=0.1 0.126 ± 0.01 ms 0.126 ± 0.0096 ms 0.993
intermixed_h n=1000 σ=1.0 0.127 ± 0.013 ms 0.133 ± 0.0095 ms 0.952
intermixed_h n=1000 σ=10.0 0.132 ± 0.013 ms 0.134 ± 0.013 ms 0.986
intermixed_h n=1000 σ=100.0 0.143 ± 0.018 ms 0.151 ± 0.019 ms 0.951
intermixed_h n=10000 σ=0.1 1.44 ± 0.28 ms 1.37 ± 0.18 ms 1.05
intermixed_h n=10000 σ=1.0 1.38 ± 0.33 ms 1.41 ± 0.2 ms 0.982
intermixed_h n=10000 σ=10.0 1.53 ± 0.45 ms 1.52 ± 0.48 ms 1.01
intermixed_h n=10000 σ=100.0 1.61 ± 0.43 ms 1.41 ± 0.21 ms 1.14
pathological 1 0.363 ± 0.0029 μs 0.37 ± 0.0024 μs 0.981
pathological 2 0.952 ± 0.0045 μs 0.949 ± 0.0049 μs 1
pathological 3 20.8 ± 1 ns 20 ± 0.71 ns 1.04
pathological 4 1.03 ± 0.0054 μs 1.04 ± 0.0039 μs 0.986
sample n=100 σ=0.1 30.6 ± 0.81 ns 29.7 ± 1 ns 1.03
sample n=100 σ=1.0 0.034 ± 0.0022 μs 0.0339 ± 0.0022 μs 1
sample n=100 σ=10.0 23.6 ± 5.6 ns 23 ± 5.5 ns 1.02
sample n=100 σ=100.0 20.7 ± 5 ns 20 ± 4.2 ns 1.04
sample n=1000 σ=0.1 28.6 ± 5.1 ns 27.8 ± 5.7 ns 1.03
sample n=1000 σ=1.0 0.0368 ± 0.0021 μs 0.0371 ± 0.0021 μs 0.993
sample n=1000 σ=10.0 25.1 ± 5.6 ns 24.2 ± 6.3 ns 1.04
sample n=1000 σ=100.0 21 ± 4.9 ns 20 ± 4.2 ns 1.05
sample n=10000 σ=0.1 0.0373 ± 0.0011 μs 0.0369 ± 0.0012 μs 1.01
sample n=10000 σ=1.0 0.04 ± 0.0017 μs 0.0403 ± 0.0014 μs 0.993
sample n=10000 σ=10.0 25.7 ± 5.9 ns 24.2 ± 6.8 ns 1.06
sample n=10000 σ=100.0 20.8 ± 4.4 ns 20.5 ± 4 ns 1.01
update ∘ rand n=100 σ=0.1 0.103 ± 0.0023 μs 0.106 ± 0.0025 μs 0.971
update ∘ rand n=100 σ=1.0 0.11 ± 0.0029 μs 0.112 ± 0.0028 μs 0.974
update ∘ rand n=100 σ=10.0 0.137 ± 0.02 μs 0.141 ± 0.018 μs 0.969
update ∘ rand n=100 σ=100.0 0.346 ± 0.037 μs 0.324 ± 0.033 μs 1.07
update ∘ rand n=1000 σ=0.1 0.104 ± 0.0029 μs 0.106 ± 0.0028 μs 0.977
update ∘ rand n=1000 σ=1.0 0.11 ± 0.0031 μs 0.113 ± 0.0031 μs 0.978
update ∘ rand n=1000 σ=10.0 0.13 ± 0.0063 μs 0.134 ± 0.0064 μs 0.97
update ∘ rand n=1000 σ=100.0 0.343 ± 0.016 μs 0.349 ± 0.017 μs 0.982
update ∘ rand n=10000 σ=0.1 0.112 ± 0.002 μs 0.113 ± 0.0021 μs 0.983
update ∘ rand n=10000 σ=1.0 0.118 ± 0.0039 μs 0.118 ± 0.0028 μs 0.999
update ∘ rand n=10000 σ=10.0 0.127 ± 0.0024 μs 0.128 ± 0.0023 μs 0.993
update ∘ rand n=10000 σ=100.0 0.309 ± 0.0036 μs 0.316 ± 0.0048 μs 0.978
time_to_load 0.725 ± 0.019 s 0.728 ± 0.027 s 0.995

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

@LilithHafner
Copy link
Owner Author

I can't reproduce the 1.57x "constructor n=10000 σ=10.0" improvement. Locally there is no performance improvement at all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant