Prevent gaps between the global validator cache and its on-disk representation #7241
+59
−23
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue Addressed
Closes #7216
Proposed Changes
Add a new field to the cache
staged_indices
. When writing to the pubkey cache, we will always insert new validators into thestaged_indices
mapping. When we write validators to disk, we will generate the database operations fromstaged_indices
. After successful writes to disk, we will flush the validators we wrote to disk fromstaged_indices
.This should prevent us from ever having gaps between our in-memory validator cache and its on-disk representation.
To be clear, an example of a gap would be:
in-memory cache:
[1,2,3,4,5]
on-disk:
[1,2,4,5]
(3 is missing, but 4 and 5 were written to disk)If a block were to fail import for any reason, we would still have the new validators included in
staged_indices
ready to be written to disk on a subsequent block import. If the program crashes before we're able to import the block to disk, we still won't have gaps. On start-up the block will need to be imported to disk anyways which will capture whatever new validators were missed and write them to disk.With this change #7217 should technically not be needed anymore. But its a simple change and there is no harm in being defensive, so I've chosen not to revert it.