Async recipient-side of static invoice server #3618

valentinewallace · 2025-02-24T22:07:04Z

As part of being an async recipient, we need to interactively build an offer and static invoice with an always-online node that will serve static invoices on our behalf in response to invoice requests from payers.

While users could build this invoice manually, the plan is for LDK to automatically build it for them using onion messages. See this doc for more details on the protocol. Here we implement the client side of the linked protocol.

See lightning/bolts#1149 for more information on async payments.

Partially addresses #2298

valentinewallace · 2025-02-24T22:22:36Z

Will go through the commits in a bit more detail before taking this out of draft, but conceptual feedback or feedback on the protocol itself is welcome, or the way the code is organized overall. It does add a significant amount of code to ChannelManager currently.

TheBlueMatt

Hmmm, I wonder if we shouldn't allow the client to cache N offers rather than only 1. I worry a bit about the privacy implications of having One Offer that gets reused across different contexts.

lightning/src/blinded_path/message.rs

valentinewallace · 2025-02-25T19:58:11Z

Hmmm, I wonder if we shouldn't allow the client to cache N offers rather than only 1. I worry a bit about the privacy implications of having One Offer that gets reused across different contexts.

I think that makes sense, so they would interactively build and cache a few and then randomly(?) return one of them on get_cached_async_receive_offer?

It seems reasonable to save for follow-up although I could adapt the AsyncReceiveOffer cache struct serialization for this now.

valentinewallace · 2025-02-25T20:05:30Z

lightning/src/ln/channelmanager.rs

+			// Expire the offer at the same time as the static invoice so we automatically refresh both
+			// at the same time.
+			let offer_and_invoice_absolute_expiry = Duration::from_secs(core::cmp::min(
+				offer_paths_absolute_expiry.as_secs(),
+				duration_since_epoch.saturating_add(STATIC_INVOICE_DEFAULT_RELATIVE_EXPIRY).as_secs()
+			));


One thing I want to address eventually (but maybe not in this PR) is that right now we cap the expiry of our offer/static invoice at 2 weeks, which doesn't work well for the "offer in Twitter bio" use case. Probably we can add something to UserConfig for this, and expose a method for users to proactively rotate their offer if it never expires?

I wonder if for that we shouldn't try to come up with a scheme to allow the offer to last longer than the static invoice? I mean ideally an offer lasts at least a few years, but it kinda could cause you just care about the storage server being reliable, you don't care much about the static invoice.

That makes sense. We could include another set of long-lived paths in the OfferPaths message that allows the recipient to refresh their invoice later while keeping the same offer [paths].

I mean maybe the OffersPath paths should just be super long-lived? I don't see a strong reason to have some concept of long-lived paths?

Even if the OfferPaths offer_paths are super long lived, we still need a way for the recipient to update their static invoice later. So the additional paths would be for that purpose, is my thinking.

Oh vs just having the original paths be long-lived? I guess we could, but it seems like we could just make all the paths long-lived?

Gotcha. In the current PR, the recipient sends PersistStaticInvoice over the reply path to the OfferPaths message, and that reply path is short-lived.

So we could make that reply path long-lived instead and have the recipient cache that reply path to update their invoice later. Just to confirm, that's what you're suggesting?

Yea, that's what I was thinking. Basically just make it a "multi-reply path"

That makes sense. It means we won't get extra message attempts over alternate blinded paths, but that might be a premature optimization anyway, hard to tell.

TheBlueMatt · 2025-02-25T22:46:31Z

I think that makes sense, so they would interactively build and cache a few and then randomly(?) return one of them on get_cached_async_receive_offer?

Yea, I dunno what to do for the fetch'er, maybe we just expose the whole list?

It seems reasonable to save for follow-up although I could adapt the AsyncReceiveOffer cache struct serialization for this now.

Makes sense, tho I imagine it would be a rather trivial diff, no?

lightning/src/onion_message/messenger.rs

lightning/src/ln/channelmanager.rs

lightning/src/util/ser.rs

jkczyz · 2025-03-03T23:10:52Z

lightning/src/onion_message/async_payments.rs

+	/// Once [`Self::invoice`] has been persisted, these paths should be used to send
+	/// [`StaticInvoicePersisted`] messages to the recipient to confirm that the offer corresponding
+	/// to the invoice is ready to receive async payments.
+	pub invoice_persisted_paths: Vec<BlindedMessagePath>,


Would using the reply path be sufficient?

I think it might be, based on discussion here #3618 (comment), yeah.

I was hesitant to use a Responder param in the handler for this message because it's not intended to be used immediately or returned with .respond_with(..) like in its other uses. The intention is for servers handling this message to go persist the invoice in their db and send a separate OM later once that's done.

With that context, thoughts on this API?

fn handle_serve_static_invoice( &self, message: ServeStaticInvoice, context: AsyncPaymentsContext, reply_path: BlindedMessagePath );

I think it might be, based on discussion here #3618 (comment), yeah.

I was hesitant to use a Responder param in the handler for this message because it's not intended to be used immediately or returned with .respond_with(..) like in its other uses. The intention is for servers handling this message to go persist the invoice in their db and send a separate OM later once that's done.

Right, but that discussion is about long-lived paths, IIUC. But here it's simply an async response because of the database write.

With that context, thoughts on this API?

fn handle_serve_static_invoice( &self, message: ServeStaticInvoice, context: AsyncPaymentsContext, reply_path: BlindedMessagePath );

I think using the responder is fine. It's serializable so can be used later with ResponseInstruction::into_instructions to get MessageSendInstructions, which then can be passed to OnionMessenger::send_onion_message to send asynchronously.

Right, but that discussion is about long-lived paths, IIUC. But here it's simply an async response because of the database write.

Right sorry, I just meant that prior to this PR reply paths were used in a fairly narrow context and we're expanding that here.

I think using the responder is fine. It's serializable so can be used later with ResponseInstruction::into_instructions to get MessageSendInstructions, which then can be passed to OnionMessenger::send_onion_message to send asynchronously.

Sounds good!

jkczyz · 2025-03-03T23:30:31Z

lightning/src/offers/signer.rs

+	const IV_BYTES: &[u8; IV_LEN] = b"LDK Offer Paths~";
+	let mut hmac = expanded_key.hmac_for_offer();
+	hmac.input(IV_BYTES);
+	hmac.input(&nonce.0);
+	hmac.input(ASYNC_PAYMENTS_OFFER_PATHS_INPUT);


Do we need to include path_absolute_expiry?

I thought the nonce/IV was sufficient but I'm not certain. @TheBlueMatt would it be an improvement to commit to the expiry in the hmac? IIUC the path still can't be re-purposed...

valentinewallace · 2025-03-04T21:16:08Z

Going to base this on #3640. Will finish updating the ser macros based on those changes and push updates here after finishing some review.

lightning/src/ln/channelmanager.rs

lightning/src/offers/invoice_request.rs

lightning/src/offers/static_invoice.rs

vincenzopalazzo

Added a couple of comments that I find out while working on the CI failure in #3593

valentinewallace · 2025-05-14T19:31:59Z

Pushed some updates after moving the async receive offer cache into the new OffersMessageFlow struct added in #3639.

lightning/src/offers/async_receive_offer_cache.rs

elnosh

New to the codebase but interested in following async payments. From reading the explanation in the commit messages, the protocol/flow between the async recipient and the always-online node to build the static invoice and offer made sense. Overall the code changes look good to me.

elnosh · 2025-05-18T21:06:56Z

lightning/src/onion_message/async_payments.rs

+	fn handle_offer_paths_request(
+		&self, message: OfferPathsRequest, context: AsyncPaymentsContext,
+		responder: Option<Responder>,
+	) -> Option<(OfferPaths, ResponseInstruction)>;


I see it is similar to other message handler traits in the OnionMessenger but I was wondering why return Options in these handle_ methods instead of Results?

Good question, I wrote that code forever ago but I think it was just consistency with the other onion message handler traits at the time. Fine to switch if reviewers prefer, although I might punt since the handle_held_htlc_available instance within the async payments trait is pre-existing...

The original OffersMessageHandler handles all offers-related message types in one method where not responding with a message is typical for handling Bolt12Invoice. Similarly, a message may not have a reply path and thus wouldn't have a responder. So the interface was more general.

For handler traits that have a method per message, it may make sense to limit the interface if there is a restricted set of possibilities. That said, I'm not sure if we want to use Result as the responding behavior should really be defined by the handler. For instance, an error handling an InvoiceRequest could mean either the corresponding Offer could not be authenticated -- and thus no response should be sent -- or the sender made an error -- in which case an InvoiceError should be sent as a response.

Somewhat related: for Responder, OnionMessenger could simply not call the handler if the message doesn't have a reply path but we are expecting one. I don't have any strong opinions yet on whether Option should be used in that case.

That said, I'm not sure if we want to use Result as the responding behavior should really be defined by the handler.

In the async payments case, as well, if we receive an offer_paths message we may have enough offers cached already and choose not to respond, which is expected behavior. So the signature may end up as Result<Option<ServeStaticInvoice>, ()>. That's not necessarily bad, but it's less clean than one might assume if we switched to using a Result.

elnosh · 2025-05-18T21:08:36Z

lightning/src/onion_message/async_payments.rs

+			Self::OfferPathsRequest(_) => OFFER_PATHS_REQ_TLV_TYPE,
+			Self::OfferPaths(msg) => msg.tlv_type(),
+			Self::ServeStaticInvoice(msg) => msg.tlv_type(),
+			Self::StaticInvoicePersisted(_) => INVOICE_PERSISTED_TLV_TYPE,


Why do some use the const directly here and others get the const set through the tlv_type on the msg?

The variants that return consts correspond to messages that don't implement the OnionMessageContents trait, so they don't have the tlv_type method available. Looks like docs are a bit lacking here but the OnionMessageContents trait implementation seems to only be needed for onion messages that are sent in direct response to other onion messages.

Isn't it more consistent to implement OnionMessageContents anyway?

I don't mind doing that, may want to save for follow-up though since it's pre-existing for a bunch of other messages too. Or could add better documentation on OnionMessageContents.

valentinewallace · 2025-05-21T00:12:39Z

Rebased on the latest version of #3639

codecov · 2025-05-21T00:27:58Z

Codecov Report

Attention: Patch coverage is 17.90123% with 133 lines in your changes missing coverage. Please review.

Project coverage is 89.76%. Comparing base (7b62ea4) to head (128208f).

Files with missing lines	Patch %	Lines
lightning/src/ln/channelmanager.rs	16.12%	52 Missing ⚠️
lightning/src/onion_message/async_payments.rs	0.00%	29 Missing ⚠️
lightning/src/onion_message/functional_tests.rs	0.00%	20 Missing ⚠️
lightning/src/ln/peer_handler.rs	0.00%	17 Missing ⚠️
lightning/src/offers/async_receive_offer_cache.rs	31.81%	15 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3618      +/-   ##
==========================================
- Coverage   89.91%   89.76%   -0.15%     
==========================================
  Files         160      161       +1     
  Lines      129307   129468     +161     
  Branches   129307   129468     +161     
==========================================
- Hits       116262   116221      -41     
- Misses      10355    10544     +189     
- Partials     2690     2703      +13

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

valentinewallace · 2025-05-22T22:53:23Z

Pushed some minor fixes for CI.

valentinewallace · 2025-06-02T17:28:28Z

Updated in response to to @joostjager's comments. Link to diff

valentinewallace · 2025-06-02T17:45:37Z

Rebased on main after #3639 landed

valentinewallace · 2025-06-02T17:55:05Z

Clarified a few more docs I noticed. Link to diff

Edit: pushed another CI fix

joostjager

Overall, this is well executed. However, as we also discussed offline, I still can't shake the feeling that this approach may be too far ahead of where we are right now.

The use case is spontaneous payments of variable amounts to offline recipients. If the current state of Lightning is LSPs with client nodes without support for blinded paths, and this is intended for that environment, then there must be a simpler way to achieve the same goal. Ultimately, the payer only needs to know the LSP node and the recipient node. With that information, they can send an onion message to the LSP, wait for the release, and then send the HTLC.

joostjager · 2025-06-03T07:05:16Z

fuzz/src/onion_message.rs

@@ -15,7 +15,8 @@ use lightning::ln::peer_handler::IgnoringMessageHandler;
 use lightning::ln::script::ShutdownScript;
 use lightning::offers::invoice::UnsignedBolt12Invoice;
 use lightning::onion_message::async_payments::{
-	AsyncPaymentsMessageHandler, HeldHtlcAvailable, ReleaseHeldHtlc,
+	AsyncPaymentsMessageHandler, HeldHtlcAvailable, OfferPaths, OfferPathsRequest, ReleaseHeldHtlc,


Commit message reads very well. Some of that valuable info might even be moved into code comments?

lightning/src/onion_message/async_payments.rs

joostjager · 2025-06-03T07:17:28Z

lightning/src/onion_message/async_payments.rs

+			Self::OfferPathsRequest(_) => OFFER_PATHS_REQ_TLV_TYPE,
+			Self::OfferPaths(msg) => msg.tlv_type(),
+			Self::ServeStaticInvoice(msg) => msg.tlv_type(),
+			Self::StaticInvoicePersisted(_) => INVOICE_PERSISTED_TLV_TYPE,


Isn't it more consistent to implement OnionMessageContents anyway?

lightning/src/ln/channelmanager.rs

joostjager · 2025-06-03T07:45:37Z

lightning/src/ln/channelmanager.rs

+	/// Will only be set if [`UserConfig::paths_to_static_invoice_server`] is set and we succeeded in
+	/// interactively building a [`StaticInvoice`] with the static invoice server.
+	#[cfg(async_payments)]
+	pub fn get_cached_async_receive_offers(&self) -> Vec<Offer> {


Tests in the follow up look good indeed.

As part of being an async recipient, we need to support interactively building an offer and static invoice with an always-online node that will serve static invoices on our behalf. Add a config field containing blinded message paths that async recipients can use to request blinded paths that will be included in their offer. Payers will forward invoice requests over the paths returned by the server, and receive a static invoice in response if the recipient is offline.

valentinewallace · 2025-06-04T20:12:02Z

Rebased after #3767 landed, haven't addressed feedback yet

Because async recipients are not online to respond to invoice requests, the plan is for another node on the network that is always-online to serve static invoices on their behalf. The protocol is as follows: - Recipient is configured with blinded message paths to reach the static invoice server - On startup, recipient requests blinded message paths for inclusion in their offer from the static invoice server over the configured paths - Server replies with offer paths for the recipient - Recipient builds their offer using these paths and the corresponding static invoice and replies with the invoice - Server persists the invoice and confirms that they've persisted it, causing the recipient to cache the interactively built offer for use At pay-time, the payer sends an invoice request to the static invoice server, who replies with the static invoice after forwarding the invreq to the recipient (to give them a chance to provide a fresh invoice in case they're online). Here we add the requisite trait methods and onion messages to support this protocol. An alterate design could be for the async recipient to publish static invoices directly without a preceding offer, e.g. on their website. Some drawbacks of this design include: 1) No fallback to regular BOLT 12 in the case that the recipient happens to be online at pay-time. Falling back to regular BOLT 12 allows the recipient to provide a fresh invoice and regain the proof-of-payment property 2) Static invoices don't fit in a QR code 3) No automatic rotation of the static invoice, which is useful in the case that payment paths become outdated due to changing fees, etc

In future commits, as part of being an async recipient, we will interactively build offers and static invoices with an always-online node that will serve static invoices on our behalf. Once an offer is built and the static invoice is confirmed as persisted by the server, we will use the new offer cache added here to save the invoice metadata and the offer in ChannelManager, though the OffersMessageFlow is responsible for keeping the cache updated. We want to cache and persist these offers so we always have them at the ready, we don't want to begin the process of interactively building an offer the moment it is needed. The offers are likely to be long-lived so caching them avoids having to keep interactively rebuilding them after every restart.

As an async recipient, we need to interactively build static invoices that an always-online node will serve to payers on our behalf. At the start of this process, we send a requests for paths to include in our offers to the always-online node on startup and refresh the cached offers when they expire.

We previously weren't dropping the cache lock if no new offers were needed, then attempted to acquire it again in a later method

We're gonna expose these for testing later so move them

valentinewallace · 2025-06-05T22:54:13Z

Mostly updated, need to fix linting. I added fixups for a few things to call attention/add a short explanation. Diff

Edit: fixed linting, diff

As an async recipient, we need to interactively build a static invoice that an always-online node will serve to payers on our behalf. As part of this process, the static invoice server sends us blinded message paths to include in our offer so they'll receive invoice requests from senders trying to pay us while we're offline. On receipt of these paths, create an offer and static invoice and send the invoice back to the server so they can provide the invoice to payers.

As an async recipient, we need to interactively build a static invoice that an always-online node will serve on our behalf. Once this invoice is built and persisted by the static invoice server, they will send us a confirmation onion message. At this time, cache the corresponding offer and mark it as ready to receive async payments.

As an async recipient, we need to interactively build offers and corresponding static invoices, the latter of which an always-online node will serve to payers on our behalf. Offers may be very long-lived and have a longer expiration than their corresponding static invoice. Therefore, persist a fresh invoice with the static invoice server when the current invoice gets close to expiration.

Over the past several commits we've implemented interactively building an async receive offer with a static invoice server that will service invoice requests on our behalf as an async recipient. Here we add an API to retrieve the resulting offers so we can receive payments when we're offline.

valentinewallace · 2025-06-05T23:05:18Z

lightning/src/offers/async_receive_offer_cache.rs

+			// Update the invoice if it expires in less than a day, as long as the offer has a longer
+			// expiry than that.


Question for reviewers: I'm not sure it makes sense to only update a static invoice if it expires in less than a day. That doesn't seem like enough time. But it only lives for 2 weeks anyway... Should we maybe make it longer lived, or replace it once it expires in a week, or?

TheBlueMatt reviewed Feb 25, 2025

View reviewed changes

lightning/src/blinded_path/message.rs Show resolved Hide resolved

valentinewallace commented Feb 25, 2025

View reviewed changes

jkczyz self-requested a review February 27, 2025 18:10

valentinewallace added the weekly goal Someone wants to land this this week label Feb 27, 2025

valentinewallace mentioned this pull request Feb 27, 2025

Static invoice server #3628

Draft

jkczyz reviewed Mar 3, 2025

View reviewed changes

valentinewallace marked this pull request as ready for review March 4, 2025 21:14

valentinewallace removed the weekly goal Someone wants to land this this week label Mar 4, 2025

This was referenced Mar 10, 2025

[RFC] Implement a way to do BOLT 12 Proof of Payment #3593

Merged

Static invoice server prefactor #3667

Merged

joostjager reviewed Mar 17, 2025

View reviewed changes

lightning/src/ln/channelmanager.rs Outdated Show resolved Hide resolved

vincenzopalazzo reviewed Mar 18, 2025

View reviewed changes

lightning/src/offers/invoice_request.rs Outdated Show resolved Hide resolved

vincenzopalazzo reviewed Mar 18, 2025

View reviewed changes

lightning/src/offers/static_invoice.rs Outdated Show resolved Hide resolved

vincenzopalazzo reviewed Mar 18, 2025

View reviewed changes

valentinewallace mentioned this pull request Apr 15, 2025

Introduce Flow utilities and OffersMessageFlow implementation #3639

Merged

valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 5cf3585 to 5455d55 Compare May 14, 2025 19:30

valentinewallace commented May 14, 2025

View reviewed changes

lightning/src/offers/async_receive_offer_cache.rs Outdated Show resolved Hide resolved

elnosh reviewed May 18, 2025

View reviewed changes

valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 5455d55 to f8023ca Compare May 21, 2025 00:11

valentinewallace force-pushed the 2025-02-static-inv-server-client branch 3 times, most recently from d1cc154 to 68fd751 Compare May 22, 2025 22:52

valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 918ed61 to ec93a15 Compare June 2, 2025 17:26

valentinewallace changed the title ~~Client-side of static invoice server~~ Async recipient-side of static invoice server Jun 2, 2025

valentinewallace force-pushed the 2025-02-static-inv-server-client branch from ec93a15 to 7823551 Compare June 2, 2025 17:44

valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 7823551 to 66d7b4f Compare June 2, 2025 17:54

valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 66d7b4f to 6adac0b Compare June 2, 2025 18:01

joostjager reviewed Jun 3, 2025

View reviewed changes

valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 6adac0b to 128208f Compare June 4, 2025 20:11

valentinewallace added 7 commits June 5, 2025 18:47

f probably should write the cache not just read it

bedf7f8

f fix deadlock

4cdac3b

We previously weren't dropping the cache lock if no new offers were needed, then attempted to acquire it again in a later method

f move consts out of struct

85eada6

We're gonna expose these for testing later so move them

f doh

df113f8

valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 128208f to 4f8d15c Compare June 5, 2025 22:48

valentinewallace added 6 commits June 5, 2025 18:58

f fix linting

be659a0

f use the right path when updating static invoices

fcbf98b

valentinewallace force-pushed the 2025-02-static-inv-server-client branch from 4f8d15c to 6fe5f86 Compare June 5, 2025 23:00

valentinewallace commented Jun 5, 2025

View reviewed changes

valentinewallace requested a review from joostjager June 5, 2025 23:05

		// Update the invoice if it expires in less than a day, as long as the offer has a longer
		// expiry than that.

Async recipient-side of static invoice server #3618

Are you sure you want to change the base?

Async recipient-side of static invoice server #3618

Uh oh!

Conversation

valentinewallace commented Feb 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

valentinewallace commented Feb 24, 2025

Uh oh!

TheBlueMatt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

valentinewallace commented Feb 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TheBlueMatt commented Feb 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

valentinewallace commented Mar 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vincenzopalazzo left a comment

Choose a reason for hiding this comment

Uh oh!

valentinewallace commented May 14, 2025

Uh oh!

Uh oh!

elnosh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

valentinewallace May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

valentinewallace commented Feb 24, 2025 •

edited

Loading

TheBlueMatt commented Feb 25, 2025 •

edited

Loading

valentinewallace May 19, 2025 •

edited

Loading

codecov bot commented May 21, 2025 •

edited

Loading

valentinewallace commented Jun 2, 2025 •

edited

Loading

valentinewallace commented Jun 5, 2025 •

edited

Loading