Block exchange optimizations #1325
Draft
+1,299
−526
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Giuliano Mega <[email protected]>
…or getBlocks; limit memory usage for fetchBatched when used as prefetcher
…ellation handling
Use probabilistic distribution based on peer quality scores, giving all peers opportunity while favoring better-performing ones. Selection probability is inversely proportional to score. Part of #974
- Reduce DefaultFetchBatch to prevent blockData explosion - Add chunked onBatch processing to prevent memory accumulation - Implement memory buffer reuse to reduce GC allocation churn - Fix short-circuit evaluation Part of #974
This prevents unbounded peer accumulation while maintaining redundancy. When peer count exceeds the maximum, the least active peers are removed from tracking to free resources. Part of #974
Achieves significant memory reduction with equivalent network performance. The reduced batch size prevents memory pressure while preserving transfer efficiency, improving overall system resource utilization. Part of #974
- Add codex_block_exchange_discovery_requests_total counter to track peer discovery frequency - Add codex_block_exchange_peer_timeouts_total counter to monitor peer reliability issues - Add codex_block_exchange_requests_failed_total counter to track request failure rates Part of #974
This prevents peers from becoming permanently invisible to block discovery when they fail to respond to WantHave requests. Part of #974
Returns false on duplicate marking attempts instead of logging errors, eliminating duplicate marking loop in blockPresenceHandler and preventing duplicate block requests across concurrent flows. Part of #974
Part of #974 Signed-off-by: Chrysostomos Nanakos <[email protected]>
Part of #974 Signed-off-by: Chrysostomos Nanakos <[email protected]>
cnanakos
force-pushed
the
feat/simple-optimizations-rebase
branch
from
46cc4ad to
372151e
Compare
cnanakos
force-pushed
the
feat/simple-optimizations-rebase
branch
11 times, most recently
from
e10d259 to
6b9ebc7
Compare
Part of #974 Signed-off-by: Chrysostomos Nanakos <[email protected]>
cnanakos
force-pushed
the
feat/simple-optimizations-rebase
branch
16 times, most recently
from
b157b2e to
85eb620
Compare
Implements delta-based WantList updates to reduce network traffic during block exchange. Only sends newly added blocks instead of resending the entire WantList on every refresh. Also some network related fixes: - Add TCP_NODELAY flag to prevent Nagle's algorithm delays - Clear sendConn on stream reset to allow garbage collection - Improve error handling in NetworkPeer.send() Part of #974 Signed-off-by: Chrysostomos Nanakos <[email protected]>
cnanakos
force-pushed
the
feat/simple-optimizations-rebase
branch
from
85eb620 to
5cf5f00
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.