Multi rfq receive (`AddInvoice` multiple hop hints) #1457

GeorgeTsagk · 2025-03-27T13:16:43Z

Description

This PR introduces the ability for tapd nodes to create invoices which involve multiple peer quotes. When the peerPubKey is left unspecified in the AddInvoice RPC we no longer return an error, but instead acquire quotes with all peers that have a valid asset channel with us.

Within this PR we also extract some rfq/liquidity related functions to the rfq package to keep rpcserver.go more clean.

Closes #1359

Based on #1423 (using that as base branch for now to avoid bloated diff)

We add a new interface to the HTLC SumAssetBalance method, which helps check the identifier of the asset against a specifier. This allows for checking asset inclusion in a group, which is a bit involved and not the responsibility of the HTLC model.

We extend the interface of the rfq Policy in order to allow the specifier checker to be involved. This extends certain checks, and allows us to use asset specifiers that only have a group key.

When intercepting an HTLC which is incoming from a btc channel and outgoing towards an asset channel, we need to create an HTLC record which reflects the asset related balance change. What really matter in these records is the total asset amount, but we also set the assetID field to be either equal to the specific asset we're sending, or the group of assets. This field does not dictate what the actual assets being sent are going to be, that will be decided later in the asset allocation process.

This allows us to get rid of a circular dependency issue that would occur in a follow up commit, where we import the adress package in the rfq package. Co-authored-by: Oliver Gugger <[email protected]>

We add the specifier checker interface to the AuxInvoiceManager too, as it is needed to validate incoming HTLCs which may use asset IDs that belong to a group, while the RFQ is based on a group key.

Adds some coverage to the invoice manager unit tests, which involve an RFQ quote over a group key, plus an HTLC with multiple asset balances, which may belong or not to the group.

We may be performing a group lookup on an asset that doesn't belong to a group. Instead of returning the error that originates from the ErrNoRows of sql we instead return a nil result, signalling that no group was found and no error occurred.

In some cases the sender of an asset HTLC may just encode the hash of the group key as the assetID of the balance in the custom records. This is done as a way to signal that any asset ID that belongs to this group may be picked to carry out the payment. This commit makes the specifier matcher aware of that case.

As mentioned in the previous commit, we occasionally want to just encode the hash of the group key as the asset ID of the balance that is encoded in the custom record. We make the ProduceHtlcExtraData hook aware of that case, which is triggered when the quote is made upon a group key and not a specific asset ID.

We have taken care of the groupkey RFQ negotiation in previous commits. All we need now to support sending a payment over a group of assets, is to propagate the user specifier groupkey to the corresponding fields.

In this commit we take the user defined group key and allow the asset specifier to be created over it. All the calls that accept it as an argument are already groupkey aware.

tapchannel/aux_traffic_shaper.go

taprpc/tapchannelrpc/tapchannel.proto

Roasbeef · 2025-03-31T23:12:25Z

rpcserver.go

@@ -6432,6 +6432,31 @@ func MarshalAssetFedSyncCfg(
 	}, nil
 }

+// marshalAssetSpecifier marshals an asset specifier to the RPC form.


So we don't allow the combo here? Of group key and an asset ID.

Roasbeef · 2025-03-31T23:13:40Z

rpcserver.go

-		}
-
-		if pass {
-			// Since the assets of the channel passed the above


Roasbeef · 2025-03-31T23:14:37Z

rfq/manager.go

@@ -1066,9 +1067,11 @@ type ChannelWithSpecifier struct {
 // each asset channel that matches the provided asset specifier.
 func (m *Manager) ComputeChannelAssetBalance(ctx context.Context,
 	activeChannels []lndclient.ChannelInfo,
-	specifier asset.Specifier) ([]ChannelWithSpecifier, error) {
+	specifier asset.Specifier) (map[route.Vertex][]ChannelWithSpecifier,


Style nit: can use a type def here to give the data structure a more descriptive name/type, and also cut down on the amt of chars needed to ref it a bit.

Roasbeef · 2025-03-31T23:31:44Z

rpcserver.go

-			acceptedQuote.AskAssetRate)
-	}
+	// Let's sort the ask rate of the quotes in ascending order.
+	sort.Slice(acquiredQuotes, func(i, j int) bool {


Solid initial strategy.

In the future, we'll also want to take into account what the allotted amount of volume is. For now, since we basically never have standing orders (we req just enough to recv), this isn't strictly required.

Roasbeef · 2025-03-31T23:32:42Z

rpcserver.go

+	// invoice amount. Since peers have varying prices for the assets, we
+	// pick the most expensive rate in order to allow for any combination of
+	// MPP shards through our set of chosen peers.
+	expensiveRate := acquiredQuotes[0].rate


Why should we pick the most expensive quote? This is effectively the same as giving the sender an option for the most expensive route in terms of fees.

Roasbeef · 2025-03-31T23:34:37Z

rpcserver.go


 	// Convert the asset amount into a fixed-point.
 	assetAmount := rfqmath.NewBigIntFixedPoint(req.AssetAmount, 0)

 	// Calculate the invoice amount in msat.
-	valMsat := rfqmath.UnitsToMilliSatoshi(assetAmount, *askAssetRate)
+	valMsat := rfqmath.UnitsToMilliSatoshi(assetAmount, *expensiveRate)


Ah ok, I think this is making a bit more sense now: ultimately we need to put some value into the invoice. I think we'll want to do another level of filtering here to ensure that we don't force the user to overpay. Or will the acceptance deviation check handle that?

Roasbeef · 2025-03-31T23:35:10Z

rpcserver.go

-		{
+	routeHints := make([]*lnrpc.RouteHint, 0)
+	for _, v := range acquiredQuotes {
+		hopHint, _, err := r.cfg.RfqManager.RfqToHopHint(


Roasbeef · 2025-03-31T23:35:23Z

rpcserver.go

-	}
-	iReq.RouteHints = []*lnrpc.RouteHint{
-		{
+	routeHints := make([]*lnrpc.RouteHint, 0)


Same here re var vs make.

lightninglabs-deploy · 2025-04-08T00:34:24Z

@GeorgeTsagk, remember to re-request review from reviewers when ready

GeorgeTsagk and others added 20 commits March 26, 2025 18:05

rfq: policies use specifier checker

4ddf460

We extend the interface of the rfq Policy in order to allow the specifier checker to be involved. This extends certain checks, and allows us to use asset specifiers that only have a group key.

multi: move rfq marshal code into rfq package

8a22cc0

This allows us to get rid of a circular dependency issue that would occur in a follow up commit, where we import the adress package in the rfq package. Co-authored-by: Oliver Gugger <[email protected]>

tapchannel: aux invoice manager uses specifier checker

3ebcce5

We add the specifier checker interface to the AuxInvoiceManager too, as it is needed to validate incoming HTLCs which may use asset IDs that belong to a group, while the RFQ is based on a group key.

tapchannel: add group key tests to AuxInvoiceManager

b61c791

Adds some coverage to the invoice manager unit tests, which involve an RFQ quote over a group key, plus an HTLC with multiple asset balances, which may belong or not to the group.

taprpc: add groupkey to SendPaymentRequest

49a0de8

rpcserver: add asset specifier marshaller

5a1ab5f

rpcserver: SendPayment uses groupkey

a8ebcaf

We have taken care of the groupkey RFQ negotiation in previous commits. All we need now to support sending a payment over a group of assets, is to propagate the user specifier groupkey to the corresponding fields.

taprpc: add groupkey to AddInvoice

a486a7b

rpcserver: AddInvoice uses groupkey

99e85a7

In this commit we take the user defined group key and allow the asset specifier to be created over it. All the calls that accept it as an argument are already groupkey aware.

multi: move rfq channel helpers to rfq manager

46a2f28

rfq+rpcserver: RfqChannel returns map of peers to channels

f5f7ae4

rfq: add RfqToHopHint helper in manager

b4eeae1

rpcserver: add AcquireBuyOrder helper

ea6446a

rpcserver: AddInvoices supports multi-rfq

f97fef1

taprpc: update AddInvoice documentation

c5d02b1

levmi assigned GeorgeTsagk Mar 27, 2025

GeorgeTsagk force-pushed the taprpc-groupkey-support branch from 99e85a7 to e2e8e47 Compare March 31, 2025 19:19

Roasbeef requested changes Mar 31, 2025

View reviewed changes

GeorgeTsagk force-pushed the taprpc-groupkey-support branch from e2e8e47 to e9e1de8 Compare April 4, 2025 13:13

Base automatically changed from taprpc-groupkey-support to main April 4, 2025 14:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multi rfq receive (`AddInvoice` multiple hop hints) #1457

Multi rfq receive (`AddInvoice` multiple hop hints) #1457

GeorgeTsagk commented Mar 27, 2025

Roasbeef Mar 31, 2025

Roasbeef Mar 31, 2025

Roasbeef Mar 31, 2025

Roasbeef Mar 31, 2025

Roasbeef Mar 31, 2025

Roasbeef Mar 31, 2025

Roasbeef Mar 31, 2025

Roasbeef Mar 31, 2025

lightninglabs-deploy commented Apr 8, 2025

Multi rfq receive (AddInvoice multiple hop hints) #1457

Are you sure you want to change the base?

Multi rfq receive (AddInvoice multiple hop hints) #1457

Conversation

GeorgeTsagk commented Mar 27, 2025

Description

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lightninglabs-deploy commented Apr 8, 2025

Multi rfq receive (`AddInvoice` multiple hop hints) #1457

Multi rfq receive (`AddInvoice` multiple hop hints) #1457