Support for reconcile confirmation against the in-memory canonical chain

[Chengxuan]

Requires hyperledger/firefly-transaction-manager#148

This PR introduced a new method, ReconcileConfirmationsForTransaction, which can be used to directly calculate the confirmation map of a given transaction.

The existing confirmation logic in FFTM has already done a great job in the following areas:

1. Reporting confirmation progress gradually as new blocks are being indexed.
2. Reuse the in-memory partial chain to build up blocks efficiently without excessive requests to the JSON RPC endpoint.
3. Emitting notifications until a transaction is confirmed.

Therefore, this PR was built on top of it with the following additional features and refinements:

1. Co-locate and give access to the in-memory partial chain inside the connector to improve efficiency (avoid building another in-memory partial chain)
2. Adds early detection of chain instability (blocks in the in-memory partial chain diverge from the JSON-RPC endpoint blocks) by walking back from the first in-memory chain element with minimal JSON-RPC calls.
3. Provides a callable function for building confirmations, removing the need for the connector to track transactions awaiting confirmation notifications.

Algorithm is peer-reviewed with @hosie , a summary can be found here: #174 (comment)

[EnriqueL8]

Thanks, looks really good - a few questions and suggestions but thanks for all the comments in the flow

[hosie]

@Chengxuan thanks for walking me through this. Here is the diagram that I sketched as we discussed it. Wanted to share it here with a couple of notes to verify my understanding

Where the main intent of the algorithm is highlighted within the grey box - i.e. to construct a contiguous linked list of blocks by chopping away at the end of the confirmations queue, filling the gap with a block from the "canonical chain or directly from the blockchain and verifying it as a valid link in the linked list.

Prereqs for entering this grey box is (and the purpose of the steps before this box)

• we have knowledge of the block that proposes transaction as per the current live view of the network's canonical chain ( named txBlock)
• we have a valid linked list of blocks of length at least 1, that starts with the transaction block (named confirmation-queue
• we have an in memory snapshot of a sliding window of the head of the networks canonical chain ( named canonical-chain)
• there is no overlap between confirmation-queue and canonical-chain. If there was any overlap, then we have spliced them buy trimming off the end of confirmation-queue rather than trimming the beginning of canonical-chain because if we do discover subsequently that the result of the splice is not a valid linkable list, then we assume canonical-chain is correct.

The early steps in the diagram ....

are not essential to the functional behaviour of the algorithm but are an optimization to allow us to end early in the case where we have already reached sufficient level of confirmation but previous iterations of the algorithm did not conclude that. Most likely because the config has been altered (by reducing the number of required confirmations) between 2 iterations.

Have I captured a correct understanding? If so, hope this write up helps explain it to others and maybe helps inform some code structure, and naming / refactoring

[Chengxuan]

Thanks @hosie for sharing the diagram. The description makes sense and is an accurate summary. I renamed canoncial-chain to InMemoryPartialChain in the last commit as per the discussion.

For the diagram, I added some annotations to give others more context.

[peterbroadhurst]

I like clarity that is is considered an error condition.

[peterbroadhurst]

I did a code test to confirm this panics if the list is empty - couldn't see any comments for why it's guaranteed not to be emby

bl.canonicalChain.Back().Value

[Chengxuan]

Will fix the panic here. Thanks for spotting it