Offchain bilinear accumulator benchmarks

[jmagan]

This is related to issue #1666.

The benchmark is created in the micro benchmark of the hydra-node package. The current benchmark is parameterized; by default it tests a membership proof of 50 TxOuts over 10,000 accumulated elements. It also benchmarks the creation of the CRS for the max elements that the accumulator holds.

Benchmark micro: RUNNING...
benchmarking BLS Accumulator/Generate CRS G1
time                 821.2 ms   (784.6 ms .. 871.0 ms)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 819.6 ms   (809.6 ms .. 829.6 ms)
std dev              12.07 ms   (5.033 ms .. 14.24 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking BLS Accumulator/Generate CRS G2
time                 1.591 s    (1.516 s .. 1.656 s)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 1.600 s    (1.589 s .. 1.617 s)
std dev              16.33 ms   (3.633 ms .. 20.66 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking BLS Accumulator/Membership Benchmark G1
time                 83.65 ms   (82.00 ms .. 84.75 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 84.45 ms   (83.81 ms .. 85.03 ms)
std dev              953.9 μs   (662.9 μs .. 1.380 ms)

benchmarking BLS Accumulator/Membership Benchmark G2
time                 121.7 ms   (116.0 ms .. 128.8 ms)
                     0.998 R²   (0.997 R² .. 1.000 R²)
mean                 115.8 ms   (114.4 ms .. 118.9 ms)
std dev              3.155 ms   (1.531 ms .. 4.594 ms)
variance introduced by outliers: 11% (moderately inflated)

Benchmark micro: FINISH

We need to do some tasks before merging:

• Change cabal.project dependencies to point CHaP.
• Create a proper nf function for the Point1 and Point2 type.

[github-actions]

Transaction cost differences

No cost or size differences found

[github-actions]

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2025-10-23 08:56:04.770393661 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	c8a101a5c8ac4816b0dceb59ce31fc2258e387de828f02961d2f2045	2652
νCommit	61458bc2f297fff3cc5df6ac7ab57cefd87763b0b7bd722146a1035c	685
νHead	a1442faf26d4ec409e2f62a685c1d4893f8d6bcbaf7bcb59d6fa1340	14599
μHead	fd173b993e12103cd734ca6710d364e17120a5eb37a224c64ab2b188*	5284
νDeposit	ae01dade3a9c346d5c93ae3ce339412b90a0b8f83f94ec6baa24e30c	1102

• The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

Init transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5837	10.93	3.49	0.52
2	6038	13.01	4.14	0.55
3	6236	14.50	4.58	0.57
5	6643	18.64	5.88	0.64
10	7648	29.31	9.25	0.79
43	14282	99.16	31.00	1.80

Commit transaction costs

This uses ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	558	2.44	1.16	0.20
2	742	3.38	1.73	0.22
3	920	4.36	2.33	0.24
5	1280	6.41	3.60	0.28
10	2170	12.13	7.25	0.40
54	10069	98.61	68.52	1.88

CollectCom transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	525	24.42	7.12	0.42
2	113	636	33.32	9.64	0.52
3	170	747	40.00	11.63	0.59
4	226	858	49.13	14.21	0.69
5	283	974	58.35	16.85	0.79
6	337	1081	72.27	20.65	0.93
7	394	1192	81.02	23.08	1.02
8	451	1303	95.41	26.86	1.17
9	505	1414	93.15	26.72	1.16

Cost of Increment Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	1804	23.92	7.60	0.48
2	1944	25.51	8.70	0.50
3	2065	27.31	9.86	0.53
5	2389	30.88	12.21	0.59
10	3174	40.76	18.29	0.75
44	7811	98.79	57.08	1.70

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	627	22.84	7.38	0.42
2	783	24.32	8.46	0.44
3	885	25.16	9.35	0.46
5	1183	29.97	12.04	0.53
10	1872	36.67	17.22	0.65
39	6275	92.97	52.20	1.55

Close transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	662	29.13	8.90	0.48
2	801	30.98	10.08	0.51
3	1029	34.18	11.67	0.56
5	1212	34.18	13.00	0.57
10	2121	48.89	20.48	0.79
36	5911	97.04	51.31	1.56

Contest transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	628	33.15	9.95	0.52
2	867	36.60	11.61	0.57
3	1068	39.34	13.05	0.61
5	1285	42.64	15.28	0.66
10	2195	55.43	22.24	0.86
28	4536	94.00	44.84	1.43

Abort transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5820	27.05	9.08	0.69
2	5986	37.09	12.51	0.80
3	6136	45.80	15.45	0.90
4	6202	51.69	17.35	0.97
5	6388	64.73	21.75	1.11
6	6559	73.65	24.83	1.21
7	6636	76.89	25.95	1.25
8	6723	89.05	29.90	1.38
9	7063	99.10	33.33	1.50

FanOut transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTXO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
10	0	0	5835	19.63	6.56	0.61
10	1	57	5868	20.34	6.91	0.62
10	10	567	6171	38.18	14.00	0.83
10	30	1709	6856	80.48	30.61	1.32
10	39	2223	7162	99.12	37.95	1.54

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2025-10-23 09:01:55.633797092 UTC

Baseline Scenario

Number of nodes	1
Number of txs	300
Avg. Confirmation Time (ms)	5.469556286
P99	7.61457138ms
P95	6.7199234ms
P50	5.2512360000000005ms
Number of Invalid txs	0

Memory data

Time	Used	Free
2025-10-23 09:00:34.636777379 UTC	1441M	4588M
2025-10-23 09:00:35.636674229 UTC	1449M	4580M
2025-10-23 09:00:36.636677432 UTC	1472M	4556M
2025-10-23 09:00:37.63673253 UTC	1518M	4484M
2025-10-23 09:00:38.636699486 UTC	1535M	4465M
2025-10-23 09:00:39.63670152 UTC	1550M	4445M
2025-10-23 09:00:40.636704599 UTC	1549M	4444M
2025-10-23 09:00:41.63660759 UTC	1549M	4444M
2025-10-23 09:00:42.636630544 UTC	1549M	4444M
2025-10-23 09:00:43.636689826 UTC	1549M	4444M
2025-10-23 09:00:44.636671554 UTC	1553M	4440M
2025-10-23 09:00:45.636756074 UTC	1553M	4440M
2025-10-23 09:00:46.636746362 UTC	1553M	4440M
2025-10-23 09:00:47.636653192 UTC	1553M	4440M
2025-10-23 09:00:48.636747769 UTC	1553M	4440M
2025-10-23 09:00:49.636644778 UTC	1553M	4440M
2025-10-23 09:00:50.636702396 UTC	1553M	4440M
2025-10-23 09:00:51.636671426 UTC	1553M	4440M
2025-10-23 09:00:52.636611435 UTC	1553M	4440M
2025-10-23 09:00:53.636686398 UTC	1553M	4440M
2025-10-23 09:00:54.63665066 UTC	1552M	4440M
2025-10-23 09:00:55.63660042 UTC	1552M	4440M
2025-10-23 09:00:56.636704228 UTC	1552M	4440M
2025-10-23 09:00:57.636609446 UTC	1552M	4440M
2025-10-23 09:00:58.636634786 UTC	1552M	4440M
2025-10-23 09:00:59.63666358 UTC	1552M	4440M
2025-10-23 09:01:00.636639328 UTC	1552M	4440M
2025-10-23 09:01:01.636739761 UTC	1555M	4437M

Three local nodes

Number of nodes	3
Number of txs	900
Avg. Confirmation Time (ms)	41.505281811
P99	78.29946810999999ms
P95	66.39104054999997ms
P50	38.7159535ms
Number of Invalid txs	0

Memory data

Time	Used	Free
2025-10-23 09:01:13.61516112 UTC	1450M	4580M
2025-10-23 09:01:14.615089919 UTC	1452M	4578M
2025-10-23 09:01:15.615022498 UTC	1468M	4562M
2025-10-23 09:01:16.615125916 UTC	1630M	4316M
2025-10-23 09:01:17.615014468 UTC	1645M	4301M
2025-10-23 09:01:18.616349032 UTC	1689M	4256M
2025-10-23 09:01:19.615149528 UTC	1701M	4241M
2025-10-23 09:01:20.61685198 UTC	1735M	4196M
2025-10-23 09:01:21.615826446 UTC	1739M	4181M
2025-10-23 09:01:22.616761238 UTC	1761M	4149M
2025-10-23 09:01:23.616793363 UTC	1765M	4134M
2025-10-23 09:01:24.617819308 UTC	1777M	4112M
2025-10-23 09:01:25.616827027 UTC	1788M	4094M
2025-10-23 09:01:26.615634236 UTC	1804M	4072M
2025-10-23 09:01:27.616439504 UTC	1807M	4063M
2025-10-23 09:01:28.617115653 UTC	1812M	4052M
2025-10-23 09:01:29.615752063 UTC	1814M	4043M
2025-10-23 09:01:30.616420314 UTC	1815M	4035M
2025-10-23 09:01:31.615035608 UTC	1827M	4016M
2025-10-23 09:01:32.61504836 UTC	1848M	3993M
2025-10-23 09:01:33.615020229 UTC	1848M	3993M
2025-10-23 09:01:34.615023564 UTC	1848M	3993M
2025-10-23 09:01:35.615170871 UTC	1848M	3993M
2025-10-23 09:01:36.615031712 UTC	1853M	3987M
2025-10-23 09:01:37.615015543 UTC	1853M	3987M
2025-10-23 09:01:38.615016607 UTC	1853M	3987M
2025-10-23 09:01:39.615039584 UTC	1853M	3987M
2025-10-23 09:01:40.615025853 UTC	1853M	3987M
2025-10-23 09:01:41.615026481 UTC	1852M	3987M
2025-10-23 09:01:42.615067537 UTC	1856M	3983M
2025-10-23 09:01:43.615028313 UTC	1856M	3983M
2025-10-23 09:01:44.615030801 UTC	1856M	3983M
2025-10-23 09:01:45.615036104 UTC	1855M	3983M
2025-10-23 09:01:46.61506291 UTC	1855M	3983M
2025-10-23 09:01:47.615044826 UTC	1855M	3983M
2025-10-23 09:01:48.615028868 UTC	1855M	3983M
2025-10-23 09:01:49.615031239 UTC	1855M	3983M
2025-10-23 09:01:50.615072562 UTC	1854M	3983M
2025-10-23 09:01:51.615035459 UTC	1854M	3983M
2025-10-23 09:01:52.615022345 UTC	1854M	3983M
2025-10-23 09:01:53.615022672 UTC	1854M	3983M
2025-10-23 09:01:54.615062189 UTC	1854M	3983M

[noonio]

Thank you @jmagan ; I think in the end we followed your logic here, but adapted it for UTxOs; see #2346.

Thank you for working on this!