resolve conflict #1
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Description Allow reading config path from env.
Adds bundle hash to redistribution output
output json diff:
```
< "Bundle": {
< "uuid": "$UUID",
< "hash": "$HASH"
< }
---
> "Bundle": "$UUID"
```
1.79 no longer compiles.
## π Summary This updates sha3-asm / keccak-asm deps. ## π‘ Motivation and Context `0.1.4` includes fixes which make keccak-asm work on more platforms (DaniPopes/keccak-asm#5) --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary Support for the upcoming Pectra hardfork. Depends on alloy-rs/alloy#1303
β¦bots#202) ## π Summary Currently we log errors that can happen normally * profit too low (that happens if we don't have enough eth on coinbase yet) * consisent db view error (that happens if block that we are building for is proposed) ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary This PR adds the option to supply the genesis fork version from a flag instead of querying the CL nodes which might not be active yet. Closes flashbots#35 ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [x] Run `make lint` * [x] Run `make test` * [x] Added tests (if applicable)
## π Summary More mocking and debug stuff. ## π‘ Motivation and Context Allow better tests! --- ## β I have completed the following steps: * [X] Run `make lint` * [X] Run `make test` * [ ] Added tests (if applicable)
## π Summary <!--- A general summary of your changes --> ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary 1. Adds sparse trie prefetcher 2. Fixes deser. error for validation request 3. Adds root hash metrics because finalize adds up to 60% to the root hash with sparse trie and its separates these two. ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary Don't filter mempool txs form the joint contribution when redistibuting to them ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary This is pretty serious regression that I've missed while reviewing external PR on one of the reth upgrades. We need to be careful to not allow something like that slip in the future. ## π‘ Motivation and Context calling recover_signer verifies transaction signature every time we try to executing something in the vm. --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary New experimental BlockBuildingAlgorithm added! ## π‘ Motivation and Context The other algorithm was quite lonely and sad. ## β I have completed the following steps: * [X] Run `make lint` * [X] Run `make test` * [ ] Added tests (if applicable)
## π Summary Removed the sleep at the beginning of the relay submit loop since this is the responsibility of the bidding module. Replaced a 5ms polling for an await to improve bidding latency. ## π‘ Motivation and Context New root hash speed makes this 5ms latency important. --- ## β I have completed the following steps: * [X] Run `make lint` * [X] Run `make test` * [ ] Added tests (if applicable)
Closes flashbots#216 Replaces all concrete usage of `ProviderFactory` and `ProviderFactoryReopener` with generics bounded by [reth provider traits](https://github.com/paradigmxyz/reth/blob/661b260f6172c047e8530e2331b2c84141e03c2b/crates/storage/provider/src/traits/full.rs#L14-L24). This allows running rbuilder in-process using a node provider and eases swapping to a more modern provider like `BlockchainProvider` in the future. Note that not all trait methods can be implemented because `ProviderFactoryReopener` has no access to the blockchain tree (pending state etc.). ## DB Consistency Checks Consistency checks have been moved out of the rbuilder core into the trait implementation for `ProviderFactoryReopener`. Inside the trait implementation, the current head block number is not known, so the reopener cannot assume that the state of the block given - 256 blocks will be available. To not redundantly re-check the consistency of the same block, `ProviderFactoryReopener` tracks the last block verified to have consistent state and will only re-check it when it changes. This adds 1 additional read per provider call to get the head of the chain, however I believe the cost will be insignificant.
## π Summary This PR introduces a new building algorithm which parallelizes the process of building blocks. In particular, it moves the largely sequential process of processing orders, resolving conflicts, and building blocks into modular and parallel process. Each of these components now runs in parallel, allowing for continuous intake of orderflow, flexible resolution of conflicts, and building blocks with the latest flow and best conflict resolution results. It is a significant architecture change. See [run_parallel_builder](https://github.com/flashbots/rbuilder/blob/parallel-builder/crates/rbuilder/src/building/builders/parallel_builder/mod.rs#L180) for a good code entry point and the [readme.md](https://github.com/flashbots/rbuilder/blob/parallel-builder/crates/rbuilder/src/building/builders/parallel_builder/readme.md) for more architecture details. Key Changes: - Introduced a new parallel_builder module, replacing the merging_builder. This adds several components for orderflow intake, conflict identification, conflict resolution, and merging of resolved conflicts. - Added a simulation cache to optimize repeated simulations of similar order sequences. - Updated the builder configuration to use the new Parallel Builder instead of the Merging Builder. - Implemented backtest functionality for the Parallel Builder. ## π‘ Motivation and Context - Concurrency driven efficiency improvements: running significant processes in parallel is more efficient, and should lead to better blocks. Processes aren't sequential, so they aren't blocked on each other anymore. - Reuses previous work: Instead of repeating conflict resolution, we are able to reuse old resolved conflict results. - Better conflict management: We can prioritize "fast" conflict resolution tasks to get results for a conflict group to the building assembler fast, but then also queue "slow" tasks (like evolutionary algorithms) which might produce better results but are slower. ## Testing Finally, I've added a range of tests to the new algorithm. I further have tested it extensively on a local node using mempool txs. - --- ## β I have completed the following steps: * [β ] Run `make lint` * [β ] Run `make test` * [ β ] Added tests (if applicable)
- Refactors `LiveBuilderConfig::new_builder` to take a generic provider - Refactors `WalletBalanceWatcher` to take a generic provider - Adds new crate `reth-rbuilder` that runs rbuilder in-process with reth - Changes config default ports that overlap with reth
## π Summary There was a bug if the block number was 0 (block_hash got 2^64-1 as a parameter) ## π‘ Motivation and Context Broke devnet tests for Pectra upgrade. --- ## β I have completed the following steps: * [X] Run `make lint` * [X] Run `make test` * [ ] Added tests (if applicable)
## π Summary - changes path of `el_node_ipc_path` in `config-playground.toml` to use `$HOME/.playground/devnet` instead of `/tmp` (aligns with current builder-playground defaults) - adds a step in the README to replace '$HOME' with your actual home path - `$HOME` was not evaluating correctly for me (tested on my ubuntu machine), fully-qualified paths fixed it ## π‘ Motivation and Context Following the [instructions](https://github.com/flashbots/rbuilder?tab=readme-ov-file#end-to-end-local-testing) without changing anything gave me this error: ```txt Ξ΄ cargo run --bin rbuilder run config-playground.toml Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.67s Running `target/debug/rbuilder run config-playground.toml` ...(truncated) Error: No such file or directory (os error 2) Caused by: No such file or directory (os error 2) Location: /home/dev/code/rbuilder/crates/rbuilder/src/live_builder/order_input/clean_orderpool.rs:30:20 ``` The [line of code](https://github.com/flashbots/rbuilder/blob/develop/crates/rbuilder/src/live_builder/order_input/clean_orderpool.rs#L30) given by the error connects to the IPC provider, which was being initialized with an invalid path given the current instructions & defaults. ## β I have completed the following steps: * [x] Run `make lint` * [x] Run `make test` * [x] Added tests (if applicable) --------- Co-authored-by: Ferran Borreguero <[email protected]>
β¦ashbots#233) When disallowed for all `Order` varients, the coinbase account has no way to spend funds when rbuilder is building all blocks for a chain. Instead of using saturating_sub, I also tried changing `coinbase_profit` from a `U256` to an `I256`, but ran into issues where a `U256` is required to calculate stuff like `mev_gas_price` in later logic. I'd prefer that approach if anyone knows if it is feasible.
## π Summary Adds `RBUILDER_BIN` build arg to Dockerfile so that a docker image for `reth-rbuilder` can be built as well. ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary Move all the code from flashbots/eth-sparse-mpt to this repo to make reth / alloy upgrades easier. <!--- A general summary of your changes --> closes flashbots#210 ## π‘ Motivation and Context eth-sparse-mpt heavily depends on reth and alloy and this will make updates of the rbuilder dependencies easier. --- ## β I have completed the following steps: * [x] Run `make lint` * [x] Run `make test` * [x] Added tests (if applicable)
## π Summary Removes the tracing initialization when spawning an `rbuilder` handle. ## π‘ Motivation and Context Reth indeed boots up tracing itself, so this is indeed unnecessary. ## β I have completed the following steps: * [x] Run `make lint` * [x] Run `make test` * [ ] Added tests (if applicable)
β¦ container (flashbots#250) ## π Summary The addition of `default-members = ["crates/rbuilder"]` in flashbots#244 broke the ability to use `docker build --build-arg RBUILDER_BIN=reth-rbuilder` to build a `reth-rbuilder` container, this should re-enable it. ## π‘ Motivation and Context We're using the `reth-rbuilder` container in kurtosis for Pectra testing. --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable) --------- Co-authored-by: liamaharon <[email protected]>
Spin up telemetry servers for rbuilder when run in `op-rbuilder`.
## π Summary - Dependency hell. - In lots of generics (eg:ProviderFactory) the need of DB+Spec was replaced for NodeTypesWithDB. - DatabaseProviderFactory DB now is asociated type and have 2 more. In lots of places now we need Provider: BlockReader. - Lots of types moved to alloy_primitives. - encode/decode_enveloped now are XXX_2718 with minimal changes. - New alloy TrieNode::EmptyRoot . - CachedReads moved to reth::revm::cached. - AsyncStateRoot replaced by ParallelStateRoot (paradigmxyz/reth#11213). - Some HashMap/Set changed to custom versions (eg: alloy_primitives::map::HashMap). - TransactionPool needs stronger restrictions on Transaction. - Updated block finalization (pectra). - Pectra relay submition. ## π‘ Motivation and Context - We needed this for Pectra and to be up to date with reth. - My life was going so well that I needed something to balance it so I did this boring PR. ## β I have completed the following steps: * [X] Run `make lint` * [X] Run `make test` * [ ] Added tests (if applicable) --------- Co-authored-by: Liam Aharon <[email protected]>
## π Summary Closes flashbots#304 ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary Closes flashbots#318. This PR uses the block building flow to notify the order pool to clean old transactions instead of using the IPC path of the EL node. ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable) --------- Co-authored-by: Daniel Xifra <[email protected]>
## π Summary It does not seem necessary for me to run the integration tests in redact-sensitive mode. ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary Generates a random relay secret key when none is set Closes flashbots#333 ## π‘ Motivation and Context --- ## β I have completed the following steps: * [x] Run `make lint` * [x] Run `make test` * [ ] Added tests (if applicable)
## π Summary Generates a random coinbase address if none was set Closes flashbots#306 ## π‘ Motivation and Context --- ## β I have completed the following steps: * [x] Run `make lint` * [x] Run `make test` * [x] Added tests (if applicable)
Make `el_node_ipc_path` optional in config. flashbots#302 ## β I have completed the following steps: * [β ] Run `make lint` * [β ] Run `make test` * [ ] Added tests (if applicable) --------- Signed-off-by: 7suyash7 <[email protected]>
## π Summary - Removes a `redact-sensitive` check following flashbots#332 - Scales up the CI runners to solve OOM issue https://github.com/flashbots/rbuilder/actions/runs/12640958168?pr=344
## π Summary Closes flashbots#305 ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable) --------- Co-authored-by: Daniel Xifra <[email protected]>
## π Summary Simplify the playground config to include only the required fields. ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
β¦ashbots#347) Refactors the `TransactionSignedEcRecoveredWithBlobs` constructor api. ## Adds `new` constructor ```rust /// Create new with an optional blob sidecar. /// /// Warning: It is the caller's responsibility to check if a tx has blobs. /// This fn will return an Err if it is passed an eip4844 without blobs. pub fn new( tx: TransactionSignedEcRecovered, blob_sidecar: Option<BlobTransactionSidecar>, metadata: Option<Metadata>, ) -> Result<Self, TxWithBlobsCreateError> ``` ## `new_no_blobs` return val Change return type from `Option<Self>` to `Result<Self, TxWithBlobsCreateError>` to better reflect behavior ## Adds `try_from` for `TransactionSignedEcRecovered` ```rust /// Try to create a [`TransactionSignedEcRecoveredWithBlobs`] from a /// [`TransactionSignedEcRecovered`] and reth pool. /// /// The pool is required because [`TransactionSignedEcRecovered`] on its /// own does not contain blob information, it is required to fetch the blob. /// /// Unfortunately we need to pass the entire pool, because the blob store /// is not part of the pool's public api. pub fn try_from_tx_without_blobs_and_pool<V, T, S>( tx: TransactionSignedEcRecovered, pool: Pool<V, T, S>, ) -> Result<Self, TxWithBlobsCreateError> ```
## π Summary This fixes a bug where we only use 1 thread where the intention was to provision a number of threads. ## π‘ Motivation and Context This will make resolving conflicts in the parallel builder much more efficient. --- ## β I have completed the following steps: * [ β ] Run `make lint` * [ β ] Run `make test` * [ ] Added tests (if applicable)
## π Summary <!--- A general summary of your changes --> ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable) --------- Co-authored-by: Daniel Xifra <[email protected]>
## π Summary <!--- A general summary of your changes --> ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> close flashbots#341 --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary <!--- A general summary of your changes --> ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
β¦ashbots#339) Adds a new method to `LiveBuilder` `connect_to_transaction_pool` allowing in-process connection between the reth transaction pool and rbuilder orderpool. This fixes an issue where the orderpool does not receive any transactions that were cached on disk by reth on start up, and finally removes the requirement for an ipc connection when running in-process.
Sometimes, the Cargo.lock will differ between `BASE_SHA` and `HEAD_SHA`. This can cause the final checkout to `HEAD_SHA` to fail. This PR resolves that, by resetting any changes prior to the final checkout.
## π Summary This PR makes backtest-build-block more versatile so the chain used and orders come from an ```OrdersSource``` instead of the hardcoded mainnet we had. The original backtest_build_block behavior is now achieved using the ```OrdersSource``` ```LandedBlockFromDBOrdersSource```. A new ```OrdersSource``` ```SyntheticOrdersSource``` is provided as an example of how to run backtest-build-block with a syntetic chain and synthetic orders. ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [X] Run `make lint` * [X] Run `make test` * [ ] Added tests (if applicable) --------- Co-authored-by: sukoneck <[email protected]>
## π Summary For some reason github allowed to merge flashbots#344 with errors! Fixed those errors and also disabled roothash calculation on backtesting. ## π‘ Motivation and Context I was threatened by the the other programmers. ## β I have completed the following steps: * [X] Run `make lint` * [X] Run `make test` * [ ] Added tests (if applicable)
## π Summary Bug introduced in flashbots#327. Rbuilder would not stop with `Ctrl-C` because it would be waiting for `spawn_clean_orderpool_job` to finish. But, `spawn_clean_orderpool_job` would not handle the cancellation token. ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [x] Run `make lint` * [x] Run `make test` * [x] Added tests (if applicable) --------- Co-authored-by: Daniel Xifra <[email protected]> Co-authored-by: liamaharon <[email protected]>
## π Summary This PR introduces a payload generator for Reth that does not create multiple block building blocks but only spawns a single process. ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary Fixed serious comments errors and typos around docs files. ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary While review found the same error in function naming get_test_mutliproofs to get_test_multiproofs in .rs files. ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary <!--- A general summary of your changes --> ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
β¦ DX (flashbots#364) ## π Summary While getting flashbots/builder-playground#37 working I ran into a couple DX gotchas with the current `config-playground.toml` file. ## π‘ Motivation and Context - The `parallel` algoirthm was pegging all my CPUs at 100%. Also figured just running `mgp-ordering` is enough. - Having a random coinbase made it harder to test block building since we only build profitable blocks. IIRC we now have a setting to build always, but I still think it's cleaner to use the prefunded accounts from the playground. - Having debug logs helped a ton tracking down pectra issues, so I expect it's useful any time anyone is using the playground for development. --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary <!--- A general summary of your changes --> ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary <!--- A general summary of your changes --> ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable)
## π Summary Lots of small changes for reth1.1.5 ## π‘ Motivation and Context Critical for pectra --- ## β I have completed the following steps: * [X] Run `make lint` * [X] Run `make test` * [ ] Added tests (if applicable) --------- Co-authored-by: Ryan Schneider <[email protected]> Co-authored-by: Ferran Borreguero <[email protected]>
## π Summary <!--- A general summary of your changes --> ## π‘ Motivation and Context <!--- (Optional) Why is this change required? What problem does it solve? Remove this section if not applicable. --> --- ## β I have completed the following steps: * [ ] Run `make lint` * [ ] Run `make test` * [ ] Added tests (if applicable) --------- Co-authored-by: Ferran Borreguero <[email protected]>
## π Summary Adds execution extension for op-rbuilder for monitoring builder transactions in the block. This ingests the committed chain and emits metrics to see if the builder transaction has landed a block or not with the op-rbuilder. ## π‘ Motivation and Context Used for observability and monitoring blocks built by the op-rbuilder on optimism. --- ## β I have completed the following steps: * [x] Run `make lint` * [x] Run `make test` * [x] Added tests (if applicable)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
π Summary
π‘ Motivation and Context
β I have completed the following steps:
make lintmake test