Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent gaps between the global validator cache and its on-disk representation #7241

Open
wants to merge 3 commits into
base: release-v7.0.0
Choose a base branch
from

Conversation

eserilev
Copy link
Collaborator

@eserilev eserilev commented Apr 1, 2025

Issue Addressed

Closes #7216

Proposed Changes

Add a new field to the cache staged_indices. When writing to the pubkey cache, we will always insert new validators into the staged_indices mapping. When we write validators to disk, we will generate the database operations from staged_indices. After successful writes to disk, we will flush the validators we wrote to disk from staged_indices.

This should prevent us from ever having gaps between our in-memory validator cache and its on-disk representation.

To be clear, an example of a gap would be:

in-memory cache: [1,2,3,4,5]
on-disk: [1,2,4,5] (3 is missing, but 4 and 5 were written to disk)

If a block were to fail import for any reason, we would still have the new validators included in staged_indices ready to be written to disk on a subsequent block import. If the program crashes before we're able to import the block to disk, we still won't have gaps. On start-up the block will need to be imported to disk anyways which will capture whatever new validators were missed and write them to disk.

With this change #7217 should technically not be needed anymore. But its a simple change and there is no harm in being defensive, so I've chosen not to revert it.

@eserilev eserilev changed the title Flush Prevent gaps between the global validator cache and its on-disk representation Apr 1, 2025
@eserilev eserilev added database electra Required for the Electra/Prague fork v7.0.0 New release c. Q1 2025 bug Something isn't working work-in-progress PR is a work-in-progress labels Apr 1, 2025
@eserilev
Copy link
Collaborator Author

eserilev commented Apr 1, 2025

I think we may be able to write some tests for this change, going to explore that

@eserilev eserilev linked an issue Apr 1, 2025 that may be closed by this pull request
@michaelsproul michaelsproul removed the v7.0.0 New release c. Q1 2025 label Apr 8, 2025
@michaelsproul
Copy link
Member

Going to drop this from v7.0 seeing as the workaround fix seems to work well. We can include this in v7.1.0

@michaelsproul michaelsproul added the v7.1.0 Post-Electra release label Apr 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working database electra Required for the Electra/Prague fork v7.1.0 Post-Electra release work-in-progress PR is a work-in-progress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Global pubkey cache is not crash safe
2 participants