[#12367] test(iceberg): add a MinIO container for credential vending ITs - #12410
Open
bharos wants to merge 6 commits into
Open
[#12367] test(iceberg): add a MinIO container for credential vending ITs#12410bharos wants to merge 6 commits into
bharos wants to merge 6 commits into
Conversation
…nding ITs The credential vending integration tests are gated behind GRAVITINO_TEST_CLOUD_IT and require a real cloud account, so they do not run in CI. MinIO enforces the session policy attached to an AssumeRole request, so the privilege-to-credential mapping is observable without one. Add MinIOContainer, wire it into ContainerSuite, and add IcebergRESTMinIOTokenAuthorizationIT, which implements the cloud hooks of IcebergRESTCloudTokenAuthorizationBaseIT against MinIO. The three test methods on that base class now run through this subclass; the existing cloud-specific subclasses stay gated and unchanged.
Code Coverage Report
|
… unaffected IcebergRESTCloudTokenAuthorizationBaseIT expects its subclass to create SCHEMA_NAME but never drops it. That was harmless while no subclass ran in CI, but the Iceberg JDBC backend outlives a test class, so the schema leaked into IcebergTableAuthorizationIT, which creates the same name and failed with SchemaAlreadyExistsException. Create the schema only when absent, since the sibling does not clean up either and may run first, and drop it on teardown. Iceberg rejects a cascading drop, so the tables are cleared first.
…ver starts In deploy mode the Gravitino server resolves S3FileIO from its own classpath at startup, so calling setupCloudBundles() after super.startIntegrationTest() was too late and /v1/config failed with a 500 while the admin Iceberg catalog was being initialised. Embedded mode cannot catch this because setupCloudBundles() returns early there. Also run the schema cleanup in a try/finally. When setup failed part way through, teardown threw before super.stopIntegrationTest() and left test_metalake behind, which then broke IcebergTableAuthorizationIT.
…lassloader too The lakehouse-iceberg catalog is loaded through IsolatedClassLoader from catalogs/lakehouse-iceberg/libs, which ships iceberg-aws but no AWS SDK, so S3FileIO resolved and then failed with NoClassDefFoundError on software/amazon/awssdk. Only iceberg-rest-server/libs was being populated, so the catalog never saw the SDK. Copy gravitino-iceberg-aws-bundle into both directories, as gravitino-server-config.md and credential-vending.md prescribe. That bundle already shades the Iceberg AWS bundle and the Gravitino credential providers, so the Maven download is no longer needed and the test stops depending on network access. Also correct the setupCloudBundles() javadoc, which told subclasses to call it after super.startIntegrationTest(); that ordering is what broke deploy mode in the first place.
…subclass IcebergRESTS3TokenAuthorizationIT called setupCloudBundles() after super.startIntegrationTest(), the ordering that fails in deploy mode for the same reason the MinIO subclass did. It is gated behind GRAVITINO_TEST_CLOUD_IT so CI never caught it. Keeps the subclass consistent with the corrected javadoc on setupCloudBundles().
bharos
marked this pull request as ready for review
August 11, 2026 18:18
… the S3 subclass" This reverts commit 3ff1e48.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Adds a MinIO-backed integration test so Iceberg credential vending is covered by CI.
MinIOContainerinintegration-test-common, plusContainerSuite.startMinIOContainer()/getMinIOContainer().IcebergRESTMinIOTokenAuthorizationIT, a subclass ofIcebergRESTCloudTokenAuthorizationBaseITtaggedgravitino-docker-testrather than gated on cloud credentials, so its three test methods run in CI.Deploy mode needed two things from the base class:
setupCloudBundles()has to run beforesuper.startIntegrationTest(), because the server resolvesS3FileIOfrom its classpath while starting. The javadoc said the opposite and is corrected.catalogs/lakehouse-iceberg/libs/as well asiceberg-rest-server/libs/. The catalog is loaded throughIsolatedClassLoaderand shipsiceberg-awswithout the AWS SDK, soS3FileIOresolved and then failed withNoClassDefFoundError. This is the layout described ingravitino-server-config.mdandcredential-vending.md.gravitino-iceberg-aws-bundleis built locally and shades both the Iceberg AWS bundle and the Gravitino credential providers, so nothing is downloaded at test time.Why are the changes needed?
Credential vending ITs are gated by
@EnabledIfEnvironmentVariable(named = "GRAVITINO_TEST_CLOUD_IT", matches = "true"), which nothing sets, and no workflow supplies cloud credentials. Whether a caller receives a writable or a read-only storage credential therefore has no automated protection against regressions.MinIO enforces the session policy attached to an
AssumeRolerequest, so that behaviour is observable without a cloud account.Fix: #12367
Does this PR introduce any user-facing change?
No. Test only.
How was this patch tested?
New test, run locally in both modes:
All three tests pass in each mode: a caller with
MODIFY_TABLEcan write with the vended credential, and a caller with onlySELECT_TABLEis rejected at the storage layer when it attempts to write.