Fullnodes to publish data columns from EL getBlobs
#7258
Conversation
jimmygchen
added
ready-for-review
| let columns_to_publish = columns | ||
| .into_iter() | ||
| .filter(|c| custody_columns.contains(&c.index)) | ||
| .collect(); | ||
| self_cloned.publish_data_columns_gradually(columns_to_publish, block_root); |
There was a problem hiding this comment.
One small optimization here might be to skip the filtering if self.network_globals.is_supernode()?
There was a problem hiding this comment.
Updated in 0c60a36
I was a bit tempted to remove the is_supernode function - with the introduction of validator custody, some full nodes now performs the same tasks as supernodes - including distributed blob publishing and reconstruction, but I'll leave this change out of this PR.
There was a problem hiding this comment.
With validator custody, this becomes a bit complicated - the node should be publishing the columns that it imported into its database - so we need to be using the custody_group_count tied to the Rpc or gossip verified beacon block instead of using the NetworkGlobals, because CGC is now dynamic:
However, there are a few moving pieces and this wiring hasn't been fully set up, I think it's best to wait until the following PR is completed:
jimmygchen
force-pushed
the
distributed-publishing-full-nodes
branch
from
8a8adb3 to
0c60a36
Compare
jimmygchen
added
ready-for-merge
Issue Addressed
Previously only supernode contributes to data column publishing in Lighthouse.
Recently we've updated the spec to have full nodes publishing data columns as well, to ensure all nodes contributes to propagation.
This also prevents already imported data columns from being imported again (because we don't "observe" them), and ensures columns that are observed in the gossip seen cache are forwarded to its peers, rather than being ignored.