Encrypt payment_metadata when we build the payment secret#4628
Encrypt payment_metadata when we build the payment secret#4628TheBlueMatt wants to merge 3 commits into
payment_metadata when we build the payment secret#4628Conversation
In 657ac8f we started committing to the `payment_metadata` in the `payment_secret`. We'd largely assumed that downstream code could simply encrypt the `payment_metadata` itself before passing it to `lightning` and decrypt before reading it from `lightning`. However, this presents a challenge - we'd very much love for that downstream code to avoid adding any extra bytes to its `payment_metadata` if at all possible, but it doesn't have a great way to get a decent IV without simply shoving it in the encrypted `payment_metadata`. Instead, here, we encrypt and decrypt the `payment_metadata` internally in `lightning`. This allows us to reuse the IV that is used for `lightning`-generated `payment_hash`es as the IV for the encrypted `payment_metadata` as well. Sadly, we don't have any similar IV for user-provided `payment_hash`es. In that case, we simply accept the limitations and document that users must avoid encrypting multiple `payment_metadata`s for payments with the same `payment_hash`. This avoids padding the size of the `payment_metadata` and should generally not be a material concern - `payment_hash` reuse should generally not exist anyway, and if it does it should only be in cases where its "the same payment" being retried after failure, at which point `payment_metadata` should hopefully be the same.
|
👋 Thanks for assigning @joostjager as a reviewer! |
|
I've completed a thorough re-review of the entire diff. All issues I identified were already flagged in my prior review. No new issues found. Review SummaryAll previously identified issues remain applicable:
Verification of crypto correctness
|
| } | ||
|
|
||
| if let Some(metadata) = payment_metadata { | ||
| ChaCha20::new_from_block( |
There was a problem hiding this comment.
How about following the PaymentMetadata / EncryptedPaymentMetadata state pattern we introduced in lightningdevkit/ldk-node#899?
We intentionally did that to improve readability and to use the type system to ensure we can't ever leak an unencrypted raw Vec<u8> into the metadata field.
There was a problem hiding this comment.
Hmm, I'm not seeing much opportunity to do this. lightning-invoice can't switch types as it has to handle counterparty data, so we have to make it a Vec<u8> again almost immediately. We could do it in PendingHTLCRouting::Receive/ReceiveKeysend but the structure in process_receive_htlcs is a bit annoying and I'm not entirely clear its worth it just for the inbound edge.
There was a problem hiding this comment.
Hmm, okay, I think I would still prefer a bit more structured/typed approach but at the very least it would be good to make this some dedicated utility methods, if only to isolate all the unwraps in a single place rather than sprinkling them everywhere (and reviewers getting used to reading "unwrap").
| iv_bytes.copy_from_slice(&rand_bytes[..IV_LEN]); | ||
|
|
||
| if let Some(metadata) = payment_metadata.as_mut() { | ||
| ChaCha20::new_from_block( |
There was a problem hiding this comment.
This and the hash-based one are duplicated. Can it be extracted to helpers?
There was a problem hiding this comment.
Hmm, we do it in a lot of places across the codebase, maybe its best if we do that separately.
There was a problem hiding this comment.
This and the hash-based one are duplicated. Can it be extracted to helpers?
Agree, essentially the same concern as above: #4628 (comment)
| /// validation. Note that because we do not store an IV in the payment secret, and to avoid adding | ||
| /// any additional overhead in the data in the payment onion, the encrypted payment metadata *only | ||
| /// uses the payment hash as an IV*. Thus, you must never encrypt multiple `payment_metadata`s using | ||
| /// the same `payment_hash`! |
There was a problem hiding this comment.
Just to check, is there really no way that a new invoice is generated with the same hash? Perhaps when the old one expires? Or in some uncommon swap protocol?
There was a problem hiding this comment.
I think its unlikely, but yea maybe its not worth saving the 16 bytes. I went ahead and added the IV.
| metadata_iv_hmac.input(&info_bytes); | ||
| metadata_iv_hmac.input(&payment_hash.0); | ||
| metadata_iv_hmac.input(&(metadata.len() as u64).to_le_bytes()); | ||
| let metadata_iv_hmac_bytes = Hmac::from_engine(metadata_iv_hmac).to_byte_array(); |
There was a problem hiding this comment.
Some explanation here why we create two IVs and how they differ would be helpful.
There was a problem hiding this comment.
Its just random now so probably doesn't need explanation.
| let (payment_hash, payment_preimage, payment_secret, encrypted_metadata) = | ||
| get_payment_hash!(nodes[1], payment_metadata.clone()); | ||
| let (mut route_0_1, ..) = get_route_and_payment_hash!(&nodes[0], &nodes[1], amt_msat); | ||
| let mut max_sized_onion = RecipientOnionFields { | ||
| payment_secret: Some(payment_secret), | ||
| payment_metadata: Some(payment_metadata.clone()), | ||
| payment_metadata: Some(encrypted_metadata), |
There was a problem hiding this comment.
Bug: create_inbound_payment_for_hash (UserPaymentHash path) appends a 16-byte IV to the encrypted metadata (metadata.extend_from_slice(&iv_bytes) in inbound_payment.rs:247). So encrypted_metadata.len() == max_metadata_len + 16.
But max_metadata_len was computed (line 75) as the exact maximum number of metadata bytes that fit in a single-hop onion payload. Putting max_metadata_len + 16 encrypted bytes in payment_metadata should cause the onion packet to exceed the 1300-byte limit, making the send_payment at line 121 fail.
The fix: subtract IV_LEN (16) from max_metadata_len so that the encrypted output (plaintext + IV) fits exactly. The same issue propagates to two_hop_metadata (line 198) and to the "too large" test (line 161).
| let (payment_hash, payment_preimage, payment_secret, encrypted_metadata) = | |
| get_payment_hash!(nodes[1], payment_metadata.clone()); | |
| let (mut route_0_1, ..) = get_route_and_payment_hash!(&nodes[0], &nodes[1], amt_msat); | |
| let mut max_sized_onion = RecipientOnionFields { | |
| payment_secret: Some(payment_secret), | |
| payment_metadata: Some(payment_metadata.clone()), | |
| payment_metadata: Some(encrypted_metadata), | |
| let (payment_hash, payment_preimage, payment_secret, encrypted_metadata) = | |
| get_payment_hash!(nodes[1], payment_metadata.clone()); | |
| let (mut route_0_1, ..) = get_route_and_payment_hash!(&nodes[0], &nodes[1], amt_msat); | |
| // Note: encrypted_metadata is max_metadata_len + IV_LEN (16) bytes for UserPaymentHash. | |
| // max_metadata_len should account for this overhead. | |
| let mut max_sized_onion = RecipientOnionFields { | |
| payment_secret: Some(payment_secret), | |
| payment_metadata: Some(encrypted_metadata), |
| if let Some(metadata) = payment_metadata.as_mut() { | ||
| let mut iv_bytes = [0 as u8; IV_LEN]; | ||
| let rand_bytes = entropy_source.get_secure_random_bytes(); | ||
| iv_bytes.copy_from_slice(&rand_bytes[..IV_LEN]); | ||
|
|
||
| ChaCha20::new_from_block( | ||
| Key::new(keys.metadata_enc_key), | ||
| Nonce::new(iv_bytes[4..16].try_into().unwrap()), | ||
| u32::from_le_bytes(iv_bytes[..4].try_into().unwrap()), | ||
| ) | ||
| .apply_keystream(metadata.as_mut_slice()); | ||
| metadata.extend_from_slice(&iv_bytes); | ||
| } |
There was a problem hiding this comment.
The 16-byte IV appended here makes create_from_hash return encrypted metadata that is 16 bytes longer than the plaintext input. This differs from create() (LdkPaymentHash path) where the encrypted metadata has the same length as the plaintext (because it reuses the payment-secret IV).
This size asymmetry isn't documented and can surprise callers who compute maximum metadata sizes for onion payloads. The docs for create_inbound_payment_for_hash (channelmanager.rs:15081) should mention the 16-byte overhead, e.g.:
Note: the returned encrypted metadata will be 16 bytes longer than the provided plaintext due to an appended encryption IV. Callers constructing invoices should account for this overhead when computing maximum metadata sizes for onion payloads.
tnull
left a comment
There was a problem hiding this comment.
CI is very failing right now.
In 657ac8f we started committing to the
payment_metadatain thepayment_secret. We'd largely assumed that downstream code could simply encrypt thepayment_metadataitself before passing it tolightningand decrypt before reading it fromlightning. However, this presents a challenge - we'd very much love for that downstream code to avoid adding any extra bytes to itspayment_metadataif at all possible, but it doesn't have a great way to get a decent IV without simply shoving it in the encryptedpayment_metadata.Instead, here, we encrypt and decrypt the
payment_metadatainternally inlightning. This allows us to reuse the IV that is used forlightning-generatedpayment_hashes as the IV for the encryptedpayment_metadataas well. Sadly, we don't have any similar IV for user-providedpayment_hashes. In that case, we simply accept the limitations and document that users must avoid encrypting multiplepayment_metadatas for payments with the samepayment_hash. This avoids padding the size of thepayment_metadataand should generally not be a material concern -payment_hashreuse should generally not exist anyway, and if it does it should only be in cases where its "the same payment" being retried after failure, at which pointpayment_metadatashould hopefully be the same.