ci: test Kerberos authenticator without Hadoop - #941
Open
GorML wants to merge 1 commit into
Open
Conversation
The Kerberos authenticator test was gated on the Hadoop/YARN CI environment removed in dask#923 and has been skipped ever since. Recreate the coverage with a self-contained setup: an ephemeral MIT Kerberos realm managed by k5test directly on the runner, exercising the real client/server SPNEGO handshake against the existing authenticator code. - tests/test_auth.py: gate the test on TEST_DASK_GATEWAY_KERBEROS and create the realm, HTTP service principal, and keytab in a fixture instead of relying on the Hadoop container's KDC - .github/workflows/test.yaml: add a kerberos-tests job - tests/requirements.txt: update the Kerberos testing docs
Collaborator
|
Thank you for adressing this, we have another ci failure on main branch - so the failures may very well be unrelated. This PR seems very reasonable though |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #924
We use dask-gateway with Kerberos authentication at my company, so I decided
to try bringing this test coverage back.
Since #923 removed the Hadoop/YARN backend from CI,
test_kerberos_authhasbeen skipped unconditionally: it was gated on
TEST_DASK_GATEWAY_YARNandrelied on the KDC, hostname and keytab of the old Hadoop container.
This PR restores real CI coverage of the Kerberos authenticator without
bringing back the unsupported Hadoop/YARN image:
k5test (a small test-only library from
the python-gssapi org), so no external KDC is needed. It is gated on a new
TEST_DASK_GATEWAY_KERBEROSenv var, and when that is set there is noskip path — missing dependencies fail the job instead of silently
skipping the test.
kerberos-testsCI job installs MIT Kerberos from apt, buildspykerberos via the
[kerberos]extras, and runs the test (~2 minutes).tests/requirements.txtupdated to match.No production code changes.
Some verification beyond CI being green here:
HTTP/<hostname>service principal on purpose makes the testfail with "Server not found in Kerberos database", so the test really
exercises the handshake and can't pass by accident.
the regular test matrix is unaffected.
Unrelated notes:
local backendandpbs/slurm backendjobs currently fail onmainfor reasons unrelated to this PR (missingdask-schedulerCLI withlatest distributed; pillow sdist build without gcc in the CI containers).
Happy to open separate issues for those.
continuous_integration/docker/hadoop/is orphaned since docs/ci: declare hadoop/yarn backend not supported, stop testing #923 and couldbe removed in a follow-up.