Commit e470758
authored
statement-store performance benchmarks (paritytech#9763)
# Description
Adds benchmarks to measure the performance of the statement-store:
- Message Exchange Scenario: interaction with one or many nodes.
- Memory Stress Test Scenario.
## Results
**Key improvements made to improve the performance:**
- [Fixed a
deadlock](paritytech#9868)
- [Increased statements
limits](paritytech#9894)
- [Improved
gossiping](paritytech#9912)
**Hardware**
All benchmarks were run on a MacBook Pro M2
### 1. Message Exchange Scenario
**Test Configuration**
- Total participants: 49_998
- Group size: 6 participants per group
- Total groups: 8_333
- Statement payload size: 512 KB
- Propagation delay: 2 seconds (empirically determined)
- Parachain network tested with: 2, 3, 6, 12 nodes
**Network Topologies**
We tested two distribution patterns:
1. **To one RPC:** All participants send statements to a single RPC node
2. **To all RPC:** Participants distribute statements across all nodes
(slower due to gossiping overhead)
**Participant Flow**
- Sends a statement with their key for an exchange session (1 sent)
- Waits 2 seconds for statement propagation
- Receives session keys from other members in the group (5 received)
- Sends statements containing a 512KB message to each member in the
group (5 sent)
- Waits 2 seconds for statement propagation
- Receives messages from other members (5 received)
- Total: 6 sent, 10 received.
**Results**
| Collators | Avg time | Max time | Memory |
| -------------- | -------- | -------- | ------ |
| **To one RPC** | | | |
| 2 | 35s | 35s | 2.1GB |
| 6 | 48s | 50s | 1.7GB |
| **To all RPC** | | | |
| 3 | 41s | 51s | 1.9GB |
| 6 | 61s | 71s | 1.4GB |
| 12 | 94s | 119s | 1.9GB |
**Observations**
- Sending to one RPC node is faster but creates a bottleneck
- Distributing across all nodes takes longer due to gossiping overhead
- More collators increase gossiping load, resulting in slower completion
times
- Memory usage per node remains around 2GB.
### 2. Memory Stress Test Scenario
**Test Configuration**
We prepared one more scenario to check how much memory nodes use with a
full store after we increased the limits. To maximize memory usage for
index usage, we submitted statements with all unique topics; other
fields (e.g., proofs) were not used.
- Total tasks: 100,000 concurrent
- Statement size: 1 KB per statement
- Network size: 6 collators
**Test Flow**
1. Spawn 100,000 concurrent tasks
2. Each task sends statements with 1KB payload to one node until store
is full
3. Statements are gossiped across the network to the other 5 collators
4. Test completes when all collator stores are full
**Results**
During the tests, each node used up to 4.5GB of memory.1 parent 2ae8298 commit e470758
File tree
5 files changed
+917
-0
lines changed- cumulus/zombienet/zombienet-sdk
- tests/zombie_ci
5 files changed
+917
-0
lines changedSome generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
24 | 25 | | |
25 | 26 | | |
| 27 | + | |
26 | 28 | | |
27 | 29 | | |
28 | 30 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
Lines changed: 148 additions & 0 deletions
Large diffs are not rendered by default.
0 commit comments