Inter-operation between libOQS and BouncyCastle for MLKEM PQC

[kaiduanx]

Hi all,

I observed some interesting behaviours when running inter-operation tests between the latest libOQS and latest BouncyCastle.

The following method is used to run the inter-operation tests.

LibOQS generates public and private keys, saves the public key with format of hex string to a file
BouncyCastle reads the public key from the file generated in step 1
BouncyCastle encapsulates the public key, generates cipher text and shared secret, saves the cipher text with format of hex string to another file
LibOQS reads the cipher text file generated in step 3, and decapsulates with private key generated in step 1, generates shared secret
Compares the two shared secrets in step 3 and step 4
Then reverses the roles between libOQS and BouncyCastle in above steps (BouncyCastle generates key pair, libOQS encapsulates, and BouncyCastle decapsulates).

Only liboqs ML-KEM implementation can inter-operate with BouncyCastle Kyber implementation.

The libOQS version is

commit open-quantum-safe/liboqs@f877812 (HEAD -> main, origin/main, origin/HEAD)
Author: Daiki Ueno [email protected]
Date: Thu Jan 30 04:15:00 2025 +0900

BouncyCastle version is 1.79/1.80.

The test code is at

https://github.com/Open-QKD-Network/oqs-bouncycastle/blob/2025/java-crypto-tools-src/gen/src/main/java/chapter15/MLKEM.java

https://github.com/Open-QKD-Network/oqs-bouncycastle/blob/2025/java-crypto-tools-src/gen/src/main/java/chapter15/Kyber.java

https://github.com/Open-QKD-Network/oqs-bouncycastle/blob/2025/example_kem.c

Do you have any insights on these inter-operation behaviours? I always assumed ML-KEM is same as Kyber.

FYI, Kyber implementation in libOQS and BouncyCastle can inter-operate without issues in 2022.

Thanks for help,

Kaiduan

FYI. I also reported the same issue to liboqs team. Please check the link open-quantum-safe/liboqs#2102

[dghgit]

So Kyber is not ML-KEM, although obviously one is a descendant of the other. I'll be interested to know what you hear back from OQS, you might want to try the latest OpenSSL instead.

[mouse07410]

OpenSSL-3.5 supports ML-KEM out of the box. OpenSSL-3.4 (and 3.2, 3.3) requires installation of oqs-provider (which in turn requires liboqs). Older versions may have bugs.

Regardless, all of the above support ML-KEM, which one should use instead of Kyber. ML-KEM interoperates with ML-KEM, Kyber - with Kyber. Not across.

Note: oqs-provider names the algorithm as, e.g., mlkem1024. OpenSSL-3.5 will name it ML-KEM-1024.

[kaiduanx]

@dghgit @mouse07410 Thanks for your comment. My question is why libOQS ML-KEM does not inter-operate with BouncyCastle ML-KEM, but libOQS can inter-operate with BouncyCastle Kyber.

LibOQS Kyber does not inter-operate with BouncyCastle Kyber either.

You know BouncyCastle implementation, what is the difference between Kyber and ML-KEM implementation in BouncyCastle?

[dghgit]

ML-KEM on Bouncy Castle has been tested against the ACVP demo system. I'd guess the LibOQS Kyber is probably based on an earlier revision and they're ML-KEM is based on the last draft.

[kaiduanx]

Per conversation from open-quantum-safe/liboqs#2102

In libOQS, ML-KEM refers to the FIPS 203 standardized version, while Kyber corresponds to the Round 3 (pre-standard) version.

@dghbk what are the standard version of Kyber and ML-KEM in BouncyCastle refers?

[dghgit]

We don't offer Kyber anymore (Kyber in the BCPQC provider is simply an alias for ML-KEM). ML-KEM is based on FIPS PUB 203 and has been tested against the ACVP demo system for correctness.

[kaiduanx]

@dghgit can you kindly take a look and review my code for ML-KEM test on

https://github.com/Open-QKD-Network/oqs-bouncycastle/blob/2025/java-crypto-tools-src/gen/src/main/java/chapter15/MLKEM.java

and point out anything wrong on this? Thanks.

[dghgit]

I'd recommend moving to MLKEM only, the Kyber classes will be deleted. Also, decide whether you want to use the JCA or the low-level API, at the moment the code appears to be mixing the two.

[kaiduanx]

@dghgit I started with ML-KEM and found out that it did not inter-operate with libOQS ML-KEM so I switched to Kyber and was surprised to know libOQS ML-KEM inter-operated with BouncyCastle Kyber :)

What I really want is simple as below.

1. API to export the ML-KEM public key as byte array (byte []). For ML-KEM-512, the length of the byte array should be 800.

2. API to construct ML-KEM public key from byte array (byte []) in step 1.

Can you point out the right way to do 1) and 2) please? Where is the BouncyCastle PQC documentation? Many thanks.

[kaiduanx]

After disabling KDF when generating cipher text and shared secret in BouncyCastle application code, libOQS and BouncyCastle ML-KEM can inter-operate.

Open-QKD-Network/oqs-bouncycastle@3807670

[dghgit]

Ah, okay, that would sense. Sorry, I probably should have thought of that myself, it's good news though the two implementations are obviously in sync, I was starting to worry. It would be worth introducing the KDF step into your example, ideally the secrets are not used directly.

[kaiduanx]

@dghgit Thank you for the awesome BouncyCastle library!

[mouse07410]

@kaiduanx apparently you made more changes than what you showed. Before the fix, both .c and .java examples compiled and ran, but produced different shared keys (due to use of KDF).

With your patches to MLKEM.java applied - both executables hang, waiting for each other:

MLKEM.java executable

/Library/Java/JavaVirtualMachines/zulu-21.jdk/Contents/Home/bin/java -javaagent:/Users/ur20980/Applications/IntelliJ IDEA Ultimate.app/Contents/lib/idea_rt.jar=63827 -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8 -Dsun.stderr.encoding=UTF-8 -classpath /Library/Java/Extensions/bctls-jdk18on-180.jar:/Library/Java/Extensions/bcpg-jdk18on-180.jar -p /Users/ur20980/IdeaProjects/MLKEM/out/production/MLKEM:/Library/Java/Extensions/bcpkix-jdk18on-180.jar:/Library/Java/Extensions/bcprov-jdk18on-180.jar:/Library/Java/Extensions/bcutil-jdk18on-180.jar -m MLKEM/MLKEM.MLKEM
Write public key to /tmp/bc-mlkem-publickey.txt
File /tmp/oqs-mlkem-ciphertext.txt is not ready, wait 1 minute
File /tmp/oqs-mlkem-ciphertext.txt is not ready, wait 1 minute
File /tmp/oqs-mlkem-ciphertext.txt is not ready, wait 1 minute

example_kem.c executable

$ ./example_kem
 BC cipher text file /tmp/bc-mlkem-ciphertext.txt does not exist, wait...
 BC cipher text file /tmp/bc-mlkem-ciphertext.txt does not exist, wait...
 BC cipher text file /tmp/bc-mlkem-ciphertext.txt does not exist, wait...
 BC cipher text file /tmp/bc-mlkem-ciphertext.txt does not exist, wait...

[kaiduanx]

@mouse07410 You need to change example_kem.c, in your above case you tried to do bc-decap and liboqs encap. But example_kem/liboqs was trying to read cipher text from BC :)

You need to change example_kem.c code to run above case.

[mouse07410]

You need to change example_kem.c code to run above case.

I see. Or change MLKEM.java...

Perhaps, it would be nice if the test "tested" both directions?