How will FreshestFirst look in the implementation?

[nfrisby]

FreshestFirst is motivated and sketched in https://iohk.io/en/research/library/papers/high-throughput-blockchain-consensus-under-realistic-network-assumptions/ as follows.

This Q&A Discussion focuses on "nodes will gossip all headers but favor downloading message bodies that have the most recent timestamps."

• "... [will] favor downloading ..." implies that it's not an option to simply download all bodies ASAP.
• This suggests that there is some sort of constrained downloading budget. (Which is explicitly listed as motivation in the other parts of the paper, for the record.)
• In the Diffusion Layer, this budget has so far been implemented (eg in the BlockFetch decision logic) by the node limiting how many bytes the node has requested on a still-open connection but not yet received, ie an upper bound on in-flight bytes.
• The first risk is that an adversarial peer will receive the request but never respond to it. Thus the node must apply a timeout in order to kill the connection if no response comes in time; closing the TCP connection soon enough (it's a race condition) is the only action that would prevent those requested bytes---wherever they currently are---from continuing to actually consume bandwidth, ie to justify removing them from the node's count of inflight bytes.
• The second risk is that an aggressive timeout might kill honest peers that are temporarily indisposed (eg running a major GC, suffering a network hiccup, etc).

So how should FreshestFirst interact with timeouts when it comes to the bandwidth resource?

Remarks:

• I'm reminded of the token bucket from the Ouroboros Genesis implementation's Limit on Patience, but I haven't considered whether that's also appropriate here. (In particular, the Ouroboros Genesis implementation is much more concerned about long-term throughput rather than immediate latency, and it can leverage the Honest Availability Assumption since it has additional Big Ledger peers. Neither of those seem to apply in Leios.)
• FreshestFirst seems potentially easier to implement for other resources, such as CPU, where it's plausible (but not necessarily trivial) to abruptly pause/cancel work. EG if the node explicitly budgets its CPU load, maybe it would cancel/pause the processing of the oldest IB being processed when a younger IB arrives.

cc @pagio @coot