Skip to content

Re-import mlkem-native and mldsa-native; drop custom meta headers#3367

Open
hanno-becker wants to merge 1 commit into
aws:mainfrom
hanno-becker:reimport-mlkem-mldsa-drop-shadow-meta
Open

Re-import mlkem-native and mldsa-native; drop custom meta headers#3367
hanno-becker wants to merge 1 commit into
aws:mainfrom
hanno-becker:reimport-mlkem-mldsa-drop-shadow-meta

Conversation

@hanno-becker

@hanno-becker hanno-becker commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

This re-imports mlkem-native and mldsa-native from their respective main branches (mlkem-native 61c831345, mldsa-native 9a180a96b).

Meta headers

This PR removes the three AWS-LC-owned "shadow" backend meta headers: ml_kem/mlkem_aarch64_meta.h, ml_dsa/mldsa_aarch64_meta.h, and ml_dsa/mldsa_x86_64_meta.h. The two AArch64 shadows (added in #3353)
duplicated the vendored meta.h to add a runtime NEON gate with C fallback (reachable via OPENSSL_armcap=0); upstream now provides this gate itself (MLK_SYS_CAP_NEON / MLD_SYS_CAP_AARCH64_NEON), making the
shadows exact duplicates. All backends consume the vendored meta.h as-is.

Full x86_64 ML-DSA backend

The x86_64 mldsa shadow trimmed the vendored meta.h to the assembly-backed subset that AWS-LC imported. The remaining AVX2 C-intrinsic operations have since been rewritten and proved in assembly upstream, so we
now import the full x86_64 backend.

Assembly local labels

Earlier revisions of this PR rewrote mldsa's local assembly labels to Lmldsa_* at import time, since identical labels in mlkem-native and mldsa-native (e.g. Lntt_layer123_start) made the FIPS delocator panic.
Upstream now namespaces local labels per project (pq-code-package/mlkem-native#1813: Lmlk_*; pq-code-package/mldsa-native#1310: Lmld_*), so the workaround is dropped.

Compatibility layer

  • Capability func: the custom configs revert to the vendored enum type (mldsa's AVX2 enumerator is now MLD_SYS_CAP_X86_64_AVX2), still mapped to AWS-LC's CRYPTO_is_{AVX2,NEON}_capable().
  • ML-DSA API: upstream dropped the sig_len argument from sign/verify (signatures are fixed-length); the ml_dsa.c wrappers are adjusted accordingly.
  • The mldsa-native update also brings upstream hardening: keygen and signing outputs are zeroized on failure.

Build

  • The mlkem assembly lists in crypto/fipsmodule/CMakeLists.txt switch to file(GLOB ... CONFIGURE_DEPENDS), matching mldsa, to absorb upstream .S renames.
  • Both importer.sh scripts make GITHUB_SERVER_URL overridable for imports from local mirrors or forks.

Verification

All imported assembly has HOL Light functional-correctness and safety proofs upstream. All assembly except the rejection samplers additionally has proofs of secret-independent timing (the samplers are
constant-time as well, but not yet proved so).


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

@hanno-becker
hanno-becker force-pushed the reimport-mlkem-mldsa-drop-shadow-meta branch from 4a4b530 to 16c4085 Compare July 23, 2026 05:04
@codecov-commenter

codecov-commenter commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.52381% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.23%. Comparing base (0f7c326) to head (8ce405c).

Files with missing lines Patch % Lines
crypto/fipsmodule/ml_dsa/mldsa/sign.c 53.19% 22 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3367      +/-   ##
==========================================
+ Coverage   78.19%   78.23%   +0.03%     
==========================================
  Files         695      696       +1     
  Lines      124192   124305     +113     
  Branches    17261    17294      +33     
==========================================
+ Hits        97115    97251     +136     
+ Misses      26154    26131      -23     
  Partials      923      923              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jakemas

jakemas commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Reviewed, this covers the same ML-DSA x86 re-import and sig_len handling as my #3357 (which it supersedes, and with a cleaner verify guard), and since it also brings mlkem-native along I'm happy to go with this PR instead.

mkannwischer added a commit to pq-code-package/mldsa-native that referenced this pull request Jul 25, 2026
Update the pinned AWS-LC integration test from v5.1.0 to v5.4.0.

The importer/glue in released AWS-LC predates the current mldsa-native
API (dropped sig_len, renamed capability enums to MLD_SYS_CAP_X86_64_AVX2
and MLD_SYS_CAP_AARCH64_NEON). aws/aws-lc#3367 is the upstream re-import
that makes AWS-LC compatible again, but it is not yet in any release, so
its integration-layer changes are applied here as a pre-import patch:
update importer.sh, ml_dsa.c, mldsa_native_config.h and
mldsa_native_backend.h, and drop the custom mldsa_x86_64_meta.h /
mldsa_aarch64_meta.h in favour of the vendored meta.h imported verbatim.

This fixes our CI integration tests against AWS-LC main.

Also exclude patch files from scripts/autogen's macro-typo check, since
they carry old macro names on their context and deletion lines.

- Fixes #1300

Signed-off-by: Matthias J. Kannwischer <matthias@zerorisc.com>
mkannwischer added a commit to pq-code-package/mldsa-native that referenced this pull request Jul 26, 2026
Update the pinned AWS-LC integration test from v5.1.0 to v5.4.0.

The importer/glue in released AWS-LC predates the current mldsa-native
API (dropped sig_len, renamed capability enums to MLD_SYS_CAP_X86_64_AVX2
and MLD_SYS_CAP_AARCH64_NEON). aws/aws-lc#3367 is the upstream re-import
that makes AWS-LC compatible again, but it is not yet in any release, so
its integration-layer changes are applied here as a pre-import patch:
update importer.sh, ml_dsa.c, mldsa_native_config.h and
mldsa_native_backend.h, and drop the custom mldsa_x86_64_meta.h /
mldsa_aarch64_meta.h in favour of the vendored meta.h imported verbatim.

This fixes our CI integration tests against AWS-LC main.

Also exclude patch files from scripts/autogen's macro-typo check, since
they carry old macro names on their context and deletion lines.

- Fixes #1300

Signed-off-by: Matthias J. Kannwischer <matthias@zerorisc.com>
Re-import mlkem-native (61c831345) and mldsa-native (9a180a96b) from
their `main` branches and remove the three AWS-LC-owned "shadow" backend
meta headers (ml_kem/mlkem_aarch64_meta.h, ml_dsa/mldsa_aarch64_meta.h,
ml_dsa/mldsa_x86_64_meta.h). The AArch64 shadows existed to add a runtime
NEON gate with C fallback; upstream now gates NEON in the vendored meta.h
itself. The x86_64 mldsa shadow trimmed the backend to the assembly-backed
subset; upstream has since converted the remaining AVX2 C-intrinsic ops to
proven assembly, so the full backend is imported and its meta.h used as-is.

Upstream now also namespaces assembly local labels per project
(mlkem-native PR#1813: `Lmlk_*`; mldsa-native PR#1310: `Lmld_*`), which
resolves the FIPS delocator duplicate-symbol clash between the two
backends at the source. The `Lmldsa_` label-prefixing workaround is
dropped from ml_dsa/importer.sh.

Adaptations: the custom configs revert to the vendored capability enum
(mldsa: `MLD_SYS_CAP_X86_64_AVX2`); ml_dsa.c is adjusted for the
`sig_len`-less upstream sign/verify API; the mlkem CMake assembly lists
use `file(GLOB ... CONFIGURE_DEPENDS)`; both importers make
`GITHUB_SERVER_URL` overridable.

All imported assembly has HOL Light functional-correctness and safety
proofs upstream. All assembly except the rejection samplers additionally
has proofs of secret-independent timing.

Signed-off-by: Hanno Becker <beckphan@amazon.co.uk>
@hanno-becker
hanno-becker force-pushed the reimport-mlkem-mldsa-drop-shadow-meta branch from 16c4085 to 8ce405c Compare July 27, 2026 04:48
@hanno-becker
hanno-becker marked this pull request as ready for review July 27, 2026 04:56
@hanno-becker
hanno-becker requested a review from a team as a code owner July 27, 2026 04:56
mkannwischer added a commit to pq-code-package/mldsa-native that referenced this pull request Jul 27, 2026
Update the pinned AWS-LC integration test from v5.1.0 to v5.4.0.

The importer/glue in released AWS-LC predates the current mldsa-native
API (dropped sig_len, renamed capability enums to MLD_SYS_CAP_X86_64_AVX2
and MLD_SYS_CAP_AARCH64_NEON). aws/aws-lc#3367 is the upstream re-import
that makes AWS-LC compatible again, but it is not yet in any release, so
its integration-layer changes are applied here as a pre-import patch:
update importer.sh, ml_dsa.c, mldsa_native_config.h and
mldsa_native_backend.h, and drop the custom mldsa_x86_64_meta.h /
mldsa_aarch64_meta.h in favour of the vendored meta.h imported verbatim.

This fixes our CI integration tests against AWS-LC main.

Also exclude patch files from scripts/autogen's macro-typo check, since
they carry old macro names on their context and deletion lines.

- Fixes #1300

Signed-off-by: Matthias J. Kannwischer <matthias@zerorisc.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants