Skip to content

Commit c6d458f

Browse files
Tobias Buckclaude
Tobias Buck
and
claude
committed
ci: run the cross-repo nornax conformance tests, which never have
`jaccpot/nornax_adapter.py` satisfies nornax's `MutualForceModel` / `FusedMutualForceModel` structurally, without importing nornax, so the dependency graph stays acyclic. Nothing checks that at import time. The only check is 24 tests in `tests/integration/test_mutual_fmm_nornax.py` and `test_mutual_distributed_nornax.py` -- the protocol match, both block-step schedules agreeing, and jaccpot's own base step against nornax's. Those 24 have skipped on every CI run since they were written. nornax is installed by no job, and `tests/conftest.py`'s `_find_sibling_checkout` resolves it from a SIBLING CHECKOUT, which exists in a local layout and never in a single-repo `actions/checkout`. So they pass locally, skip remotely, and a skipped test passes: a protocol drift in nornax would not have turned anything red. Two jobs rather than one, because they cost different things: * `test-cross-repo-nornax` -- the adapter file, one device, 25 min. Measured 137 s for its 12 tests locally. * `test-cross-repo-nornax-distributed` -- the mesh file, four forced host devices, 90 min. Both files together measured 32:48 locally with four devices throughout; subtracting the adapter file at that device count leaves ~26 min for this half. Splitting them is not tidiness. Forcing four devices makes the single-device file ~5x slower (~11 s/test without the flag, ~36 s/test with it, same tests, same box), and folding them together would hold a two-minute protocol check hostage to a 26-minute mesh suite. Each job ends with a fail-if-skipped guard, modelled on the one `test-distributed-mutual` already carries, since a green-but-vacuous job is the exact state this is meant to end. The guards key on the module-level `nornax unavailable` message, not on any SKIPPED line: the distributed file legitimately skips per-test when a kernel needs sm_80, and reading that as vacuity would fail the job over a test that could not run on a CPU box. nornax is tracked unpinned on purpose. A pinned SHA gives a reproducible build that by construction can never tell you the protocol drifted, which is the one thing these jobs are for. The cost is that a nornax push can redden this CI. The `integration` shard of `test-full` now ignores both files, so they are not collected twice -- the same convention `tests/unit/runtime` already uses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 0931c69 commit c6d458f

1 file changed

Lines changed: 135 additions & 1 deletion

File tree

‎.github/workflows/ci.yml‎

Lines changed: 135 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,14 @@ jobs:
155155
# both `-n 2` and `-n 1`. Do not group them.
156156
- shard: integration
157157
paths: tests/integration
158-
extra: --ignore=tests/integration/test_mutual_fmm_static_device.py
158+
# The two `*_nornax.py` files belong to `test-cross-repo-nornax`, which
159+
# installs nornax and guards against a vacuous run. Ignored here so they
160+
# are not collected twice -- and so this shard's log stops showing 24
161+
# module-level skips that look like a failure signal but are not.
162+
extra: >-
163+
--ignore=tests/integration/test_mutual_fmm_static_device.py
164+
--ignore=tests/integration/test_mutual_fmm_nornax.py
165+
--ignore=tests/integration/test_mutual_distributed_nornax.py
159166
- shard: mutual-static-device
160167
paths: tests/integration/test_mutual_fmm_static_device.py
161168
extra: ""
@@ -563,6 +570,133 @@ jobs:
563570
exit 1
564571
fi
565572
573+
# The cross-repo conformance tier, in TWO jobs because they cost two different
574+
# things -- and because the cheap half is the one worth failing fast on.
575+
#
576+
# jaccpot's `BlockStepFMM` satisfies nornax's `MutualForceModel` /
577+
# `FusedMutualForceModel` STRUCTURALLY: `jaccpot/nornax_adapter.py` deliberately does
578+
# not import nornax, so the dependency graph stays acyclic (Jaccpot -> Yggdrax, Nornax
579+
# standalone, ODISSEO -> both). These two files are the only thing that checks the
580+
# duck-typing still holds -- the protocol match itself, both block-step schedules
581+
# agreeing, and jaccpot's own base step matching nornax's.
582+
#
583+
# WHY THEY EXIST. Until they did, all 24 of those tests skipped on EVERY run and the
584+
# suite stayed green. nornax was installed by no job, and `tests/conftest.py`'s
585+
# `_find_sibling_checkout` resolves nornax from a SIBLING CHECKOUT -- which exists in a
586+
# local layout and never in a single-repo `actions/checkout`. So the tests passed
587+
# locally, skipped in CI, and nothing would have caught a protocol drift.
588+
test-cross-repo-nornax:
589+
name: test-cross-repo-nornax
590+
runs-on: ubuntu-latest
591+
# MEASURED: 12 tests in 137 s on a fast workstation, single device. 25 is ~10x that,
592+
# which is generous for a job whose whole point is to fail fast.
593+
timeout-minutes: 25
594+
env:
595+
JAX_ENABLE_X64: "1"
596+
JAX_PLATFORMS: cpu
597+
# NB: XLA_FLAGS is deliberately NOT set. Forcing four host devices makes this file
598+
# ~5x slower (measured: ~11 s/test without, ~36 s/test with, same 12 tests, same
599+
# box) and it needs exactly one device. The distributed half is the job below.
600+
steps:
601+
- name: Checkout
602+
uses: actions/checkout@v4
603+
- name: Setup Python
604+
uses: actions/setup-python@v5
605+
with:
606+
python-version: "3.13"
607+
cache: pip
608+
- name: Install package + dev tools + nornax
609+
# ORDER IS LOAD-BEARING, for the reason pyproject.toml already records for
610+
# yggdrax: a bare `pip install git+...` resolves JAX unconstrained, and the
611+
# `<0.11` ceiling in `pip install -e ".[dev]"` is what downgrades it back.
612+
# nornax's own floor is jax>=0.8.0, so the resulting 0.10.2 satisfies it.
613+
#
614+
# nornax is TRACKED UNPINNED on purpose. A pinned SHA gives a reproducible build
615+
# that by construction can never tell you the protocol drifted, which is the one
616+
# thing these jobs are for. The cost is that a nornax push can redden jaccpot's
617+
# CI. If that becomes disruptive, pin here and add a scheduled unpinned run.
618+
run: |
619+
python -m pip install --upgrade pip
620+
pip install git+https://github.com/TobiBu/yggdrax.git
621+
pip install git+https://github.com/TobiBu/nornax.git
622+
pip install -e ".[dev]"
623+
- name: Adapter conformance suite
624+
# `-rs` puts skip reasons in the log, which the guard step reads.
625+
run: |
626+
set -o pipefail
627+
rc=0
628+
pytest tests/integration/test_mutual_fmm_nornax.py -n 0 -rs -v \
629+
| tee pytest-cross-repo-adapter.txt || rc=$?
630+
# Exit 5 is pytest's "no tests were collected", which is what a module-level
631+
# skip produces in a job that runs ONE file. Tolerated so the guard below gets
632+
# to say WHY, with an actionable message, rather than dying on a bare exit 5.
633+
# Every other non-zero code still fails this step.
634+
if [ "$rc" != 0 ] && [ "$rc" != 5 ]; then exit "$rc"; fi
635+
- name: Fail if the suite was skipped rather than run
636+
# A skipped suite PASSES, and a green-but-vacuous job is the exact state this job
637+
# was added to end. Anchored on the module-level skip message --
638+
# `pytest.skip(f"nornax unavailable: {exc!r}", allow_module_level=True)` -- and
639+
# NOT on any SKIPPED line, so a legitimate per-test skip cannot fail the job.
640+
run: |
641+
if grep -qE "no tests ran|collected 0 items" pytest-cross-repo-adapter.txt \
642+
|| grep -qE "^SKIPPED.*nornax unavailable" pytest-cross-repo-adapter.txt; then
643+
echo "::error::the cross-repo nornax adapter suite was skipped, not run -- see the reason in the previous step"
644+
exit 1
645+
fi
646+
647+
# The distributed half, split out for the same reason `test-distributed-criterion` is:
648+
# what it costs. Its 12 tests each build their own mesh on forced host devices.
649+
#
650+
# MEASURED: the two files together, all 24 tests, took 32:48 on a fast workstation with
651+
# four forced devices throughout. Subtracting the adapter file at that device count
652+
# (~36 s/test x 12) leaves ~26 min for this half alone. Hosted runners are the slower
653+
# side of that, which is why the timeout is 90 rather than the 35 a naive reading of the
654+
# local number would suggest -- and why this is not folded into the job above, where it
655+
# would hold a 2-minute protocol check hostage to a 26-minute mesh suite.
656+
test-cross-repo-nornax-distributed:
657+
name: test-cross-repo-nornax-distributed
658+
runs-on: ubuntu-latest
659+
timeout-minutes: 90
660+
env:
661+
JAX_ENABLE_X64: "1"
662+
JAX_PLATFORMS: cpu
663+
# Below two devices these tests skip per-test rather than run.
664+
XLA_FLAGS: --xla_force_host_platform_device_count=4
665+
steps:
666+
- name: Checkout
667+
uses: actions/checkout@v4
668+
- name: Setup Python
669+
uses: actions/setup-python@v5
670+
with:
671+
python-version: "3.13"
672+
cache: pip
673+
- name: Install package + dev tools + nornax
674+
# Same ordering constraint as the job above.
675+
run: |
676+
python -m pip install --upgrade pip
677+
pip install git+https://github.com/TobiBu/yggdrax.git
678+
pip install git+https://github.com/TobiBu/nornax.git
679+
pip install -e ".[dev]"
680+
- name: Distributed conformance suite
681+
# `-n 0`: the four forced host devices live in ONE process and each test builds
682+
# its own mesh, so serial keeps the device set unambiguous.
683+
run: |
684+
set -o pipefail
685+
rc=0
686+
pytest tests/integration/test_mutual_distributed_nornax.py -n 0 -rs -v \
687+
| tee pytest-cross-repo-distributed.txt || rc=$?
688+
if [ "$rc" != 0 ] && [ "$rc" != 5 ]; then exit "$rc"; fi
689+
- name: Fail if the suite was skipped rather than run
690+
# Vacuity only -- see the note in the job above. This file legitimately skips
691+
# per-test when a kernel needs sm_80, and reading that as vacuity would fail the
692+
# job over a test that could not have run on a CPU box.
693+
run: |
694+
if grep -qE "no tests ran|collected 0 items" pytest-cross-repo-distributed.txt \
695+
|| grep -qE "^SKIPPED.*nornax unavailable" pytest-cross-repo-distributed.txt; then
696+
echo "::error::the cross-repo nornax distributed suite was skipped, not run -- see the reason in the previous step"
697+
exit 1
698+
fi
699+
566700
# The Dehnen-criterion tier, in its own job because of what it costs.
567701
#
568702
# One distributed FMM evaluation is ~190 s of COMPILATION on a forced-CPU mesh, and

0 commit comments

Comments
 (0)