First draft of NIP-44v3

[vitorpamplona]

This PR addresses 3 main problems of NIP-44v2:

1. It has a message size limit of 65Kb, which is unnecessarily small. And we need a new version to increase it. This PR raises the 65KB limit to ~4GB. 4GB is just an encoding upper limit, I don't expect us to have 4GB payloads. Discussion from this: [Nip-46] - Issue when using nip 44 for encrypting/decrypting bunker requests #1712

2. It forces the encrypting key to be the same as the event's signing key. Which forces multi-sig actors to share their main private key in order to encrypt the payload that would be later signed by the group. Decoupling singing and encryption keys, for both source and destination, is one of the goals of this version. Discussion from nip4e: decoupling encryption from identity #1647 (comment)

3. It offers no way to describe what's inside the encrypted blob before requesting the user's approval to decrypt and send the decrypted info back to the requesting application. This PR adds an alt description to allow decrypting signers to display a message and warn the user of what type of information the requesting application is receiving. The goal, for instance, is to let users block social media applications from decrypting health data and other types of information social apps should not touch. Long discussion here: Is NIP-07 decryptEvent(event) a good idea? #1439

[vitorpamplona]

@arthurfranca @mikedilger @paulmillr

[paulmillr]

Thank you for being sensible and implementing this as ver upgrade! Will review in a bit.

[paulmillr]

This seems like a complete rewrite of v2. I'm concerned about complexity and subtle bugs.

encrypt and decrypt generically consume conversation_key so that anything could be used as conversation_key.

Can't we just bump data limits and define new method to create conversation/message key?

[arthurfranca]

1. ack; 4GB seems massive though

2. I see no reason whatsoever for nip4e: decoupling encryption from identity #1647 flow to be part of nip44 spec. One previously pick the keys with whatever flow, then they use nip44 with the selected keys to encrypt/decrypt. The metadata you added could be inside nostr events so that flows other than nip4e can arise.

3. ack to AAD, though it was dismissed as impossible. Are you considering the extra base64 metadata that anyone could tamper with as AAD? I may have misunderstood this part. Without AAD I was going to use this instead that isn't as fancy as real AAD but can't be tampered with.

[vitorpamplona]

This seems like a complete rewrite of v2.

It only looks like that because of the new way to compute the conversation keys. The core encryption is exactly the same. We are adding a different encoding for the padding and the conversation key to decrypt can only be computed after decoding the payload. That's why the signature of the method has changed.

I see no reason whatsoever for #1647 flow to be part of nip44 spec.

I did try to generalize it, but I am not sure if it is possible because this setup requires the publication of keys and the use of nonces. Everything is tightly coupled together. We could move the definition of the 10044 event kind out. While that would make things more flexible (other kinds could be used to publish keys), v3 would still require some form of key-nonce storage. It doesn't make much sense to leave a direct dependency undefined.

AAD

I am not sure what classifies as "real AAD", but this approach is similar to yours but expects signers to provide an alt description for what's inside the payload. That alt is also encrypted with the payload.

[vitorpamplona]

hum... I just realized that by clearly marking which key was this encrypted for in the encoding we break the plausible deniability of NIP-17's Seals...

[mikedilger]

This is just my first few thoughts, I didn't spent much time yet.

[mikedilger]

Are people searching for these 'n' tags? If not, no need for a single letter.

[mikedilger]

would you change this 'nip44-v2' salt to 'nip44-v3'?

[mikedilger]

Indicate this is same as v2 perhaps with (identical to v2) (and to subsequent sections that haven't changed)

[mikedilger]

Indicate this is same as v2 perhaps with (identical to v2)

[mikedilger]

One other thing: I'd like to use v3 to encrypt a secret blob from the master key to each device key, as a starting point for deriving a sequence of encryption keys (according to ideas in the #1647 discussion). That means that the prose regarding which keys to select is too specific for NIP-44. NIP-44 should be just a post-key-selection encryption scheme, and not specify the key selection criteria itself.

[paulmillr]

It only looks like that because of the new way to compute the conversation keys. The core encryption is exactly the same. We are adding a different encoding for the padding and the conversation key to decrypt can only be computed after decoding the payload. That's why the signature of the method has changed.

v2 was made as generic as possible - dev can put anything into conversation_key, also anything into payload.

How about that? Just define 3 new methods:

get_key_v3(conversation_key, deriver) # executed after get_conversation_key
pack_message_v3(message, additional_data) # executed before encryption
# also maybe unpack_

and let everything else stay the same. This would make scheme composable and easy to reason about.

[arthurfranca]

I am not sure what classifies as "real AAD", but this approach is similar to yours but expects signers to provide an alt description for what's inside the payload. That alt is also encrypted with the payload.

I didn't notice the alt thing. Encrypted with payload is ok. Nice addition. Better than my string prefix/suffix that would be needed if reusing nip44v2.

"real AAD" as in somet metadata readable by anyone before decryption (not really encrypted with payload) but that should be used and match when encrypting and decrypting. Thought you were referring to the part where you concat and base64 encoding some fixed extra params.

[mikedilger]

AEAD (Authenticated Encryption with Associated Data) encrypts some data, and authenticates all the data even the data that wasn't encrypted, sometimes in a single cryptographic operation (like OCB mode that never got much use because Rogaway, my crypto professor, patented it).

[erskingardner]

👀

[vitorpamplona]

FYI: I coded this but I am not happy with it.

The main problem is that it forces all encryptions to document the source and destination of the message in the payload itself. And that means that it breaks privacy because you will always know the two keys that are talking to one another.

On NIP-17, the GiftWrap encrypts and signs with a random key to a known destination. That is not a problem for this spec since the random key is known and the destination is also already known.

But for Seals, the encryption and the event must not include a destination at all. Adding it to the NIP-44 payload breaks the message's metadata privacy if the sealed event leaks to a relay.

I don't know how to solve that yet.

[fiatjaf]

It's hard to read this entire thing. If most things are exactly the same as v2 then it's confusing and weird to have v2 all copy-pasted below. It would be much easier to just notice the differences.

[vitorpamplona]

It's hard to read this entire thing. If most things are exactly the same as v2 then it's confusing and weird to have v2 all copy-pasted below. It would be much easier to just notice the differences.

Yeah, I didn't know how do we want to manage different algorithms for different versions. Since v2 is out there and people will need to decrypt it to see the past, I thought keeping the V2 text was a safe play. But we can override it as well.

We can also redesign v3 as a wrapper around v2 itself, like @paulmillr suggested.

[fiatjaf]

I can't figure this out for myself, but this v3 would break all existing implementations, right? How are you supposed to know if you should encrypt using v2 or v3? That's why versions are stupid, you have to always break everything and that's horrible in a decentralized ecosystem like this.

I haven't tried to do it myself, but I think all the changes proposed here can be added optionally on top of v2 while keeping old v2 clients still working (even if they don't support the new features). I hope that's what you mean by making it a wrapper around v2.

[paulmillr]

@fiatjaf version is encoded in first byte. Existing clients will see this as "unsupported version" and can either show the popup to user asking to upgrade their client, or silently ignore it.

[fiatjaf]

In other words: the user will lose the message because their client won't have the new version.

And by creating pressure to always be updating to a newer encryption version we make it harder for apps to compete and centralize the ecosystem into the fewer apps that can afford to be constantly being kept up to the date with the spec changes.

This might be a big deal or may amount to nothing if we never do another version update. But in principle having this easy way to create new versions is a bad route for an open decentralized protocol.

[fiatjaf]

Another thing with this proposal -- that may or not may not be a problem (I can't decide) but it's worth noticing: if we're using an nsec to generate these keys, then we're really not decoupling encryption from identity and any compromised identity key also gives access to all encrypted messages. It's an improvement over the current state, but not really. And even though we can use some other secret other than the nsec in order to create the encryption keys that would still give us a "master" key that would have to be stored somewhere (and in the case of multisig bunkers that would have to be known by a central bunker operator).

@vitorpamplona did you think about a standardized way to distribute these generated encryption keys between apps?

[vitorpamplona]

decoupling encryption from identity and any compromised identity key also gives access to all encrypted messages.

Decoupling doesn't mean forward secrecy. We can decouple in a way that the nsec still loads everything up but the decryption keys can be exposed to clients or decouple and then have to deal with import & export a separate set of keys outside of Nostr. Because if they are inside of Nostr, there will be a way to save them even if the event is ephemeral. And that would break the desired forward secrecy.

@vitorpamplona did you think about a standardized way to distribute these generated encryption keys between apps?

In this scheme, with a non-FROST key, there is no need to distribute any keys. Each client can ask the key from the signer directly (new derive method is needed).

The main goal if this approach is to make sure the nsec alone is the only backup users need to have. If we start adding more stuff to the backup requirements in order to fully recover DMs and so on, we will be significantly changing how simple Nostr is.

[fiatjaf]

In this scheme, with a non-FROST key, there is no need to distribute any keys. Each client can ask the key from the signer directly (new derive method is needed).

That's what I'm talking about. The bunker could implement this, it would work. Although it would still be bad as the coordinator would have access to all encryption keys but there is not much way around this anyway.

The main goal if this approach is to make sure the nsec alone is the only backup users need to have. If we start adding more stuff to the backup requirements in order to fully recover DMs and so on, we will be significantly changing how simple Nostr is.

I think this is a confusing take. Nostr's simplicity was never associated with this value proposition of "always be able to recover everything with just the nsec" in my view. I think simplicity is more related to the protocol being simple to implement, that other thing is another feature and one that we never guaranteed, and we shouldn't, because ideally you won't broadcast your DMs to a thousand relays, you should keep them in 1 or 2 at most, and these can vanish and then you lose your DMs. Whatever.

[fiatjaf]

Instead of using the user's nsec directly there is an alternative that would be immediately compatible with FROST and MuSig2 bunkers without requiring them to use a "root encryption key" different than their nsec: we make the private_key in this context be equal to:

private_key = ecdh(nsec, nums_public_key)

The public key should be a NUMS otherwise it becomes possible for anyone to gather the root encryption key of anyone else. As a cool but suspiciously biased suggestion we could use the id of this event: https://njump.me/nevent1qqstnl4ddmhc0kzqpj7p543pvq9nvppc4laewc9x5ppucz7aagsa4dsppamhxue69uhkummnw3ezumt0d5pzpq35r7yzkm4te5460u00jz4djcw0qa90zku7739qn7wj4ralhe4z4huyw2, or, I don't know, something like the hash of Bitcoin block 900000.

[fiatjaf]

I actually think we don't need a NUMS key here because we hash the thing afterwards. Consult a cryptographer before taking this seriously.

[vitorpamplona]

@paulmillr, what would it take to do a quantum resistant version of NIP-44 together in some v3 version?

[paulmillr]

@vitorpamplona just sitting down and investing some time into it. ~1 week, perhaps more. Also some things are unknown.

• In classical scenario, shared key is generated using ECDH: shared = alicePub * bobPriv. It was easy to do with nostr, because it already uses these pubs/privs.
• In pq scenario, ML-KEM (768 byte keys, lattices) or HQC (2kb keys, codes) are used for KEM - instead of ECDH.
• In recommended hybrid scenario, pq is combined with classical algos.
• However, it's unclear where the "public keys" for ML-KEM would reside in, because nostr doesn't currently use them