[SPARK-56353][BUILD][TESTS][FOLLOWUP] Fix resource loading to work when resources are inside JARs#55299
Open
sarutak wants to merge 1 commit intoapache:masterfrom
Open
[SPARK-56353][BUILD][TESTS][FOLLOWUP] Fix resource loading to work when resources are inside JARs#55299sarutak wants to merge 1 commit intoapache:masterfrom
sarutak wants to merge 1 commit intoapache:masterfrom
Conversation
Use url.openStream() instead of Paths.get(url.toURI) to read classpath resources. Paths.get() throws FileSystemNotFoundException when the resource is inside a JAR file (e.g., in Maven CI), because the jar: URI scheme requires a ZipFileSystem that is not installed by default.
LuciferYang
reviewed
Apr 12, 2026
| val url = Thread.currentThread().getContextClassLoader.getResource(name) | ||
| assert(url != null, s"Resource not found: $name") | ||
| new String(Files.readAllBytes(Paths.get(url.toURI)), StandardCharsets.UTF_8) | ||
| new String(url.openStream().readAllBytes(), StandardCharsets.UTF_8) |
Contributor
There was a problem hiding this comment.
Suggested change
| new String(url.openStream().readAllBytes(), StandardCharsets.UTF_8) | |
| Utils.tryWithResource(url.openStream()) { is => | |
| new String(is.readAllBytes(), StandardCharsets.UTF_8) | |
| } |
| val url = Thread.currentThread().getContextClassLoader.getResource(name) | ||
| assert(url != null, s"Resource not found: $name") | ||
| new String(Files.readAllBytes(Paths.get(url.toURI)), StandardCharsets.UTF_8) | ||
| new String(url.openStream().readAllBytes(), StandardCharsets.UTF_8) |
| val url = Thread.currentThread().getContextClassLoader.getResource(name) | ||
| assert(url != null, s"Resource not found: $name") | ||
| new String(Files.readAllBytes(Paths.get(url.toURI)), StandardCharsets.UTF_8) | ||
| new String(url.openStream().readAllBytes(), StandardCharsets.UTF_8) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR fixes
FileSystemNotFoundExceptioninTestSpark21101Jar,TestUDTFJar, andTestHiveUdfsJarwhen running tests with Maven.These files used
Paths.get(url.toURI)to read Java source files from classpath resources. This works with SBT where resources are extracted totarget/test-classes/as regular files, but fails with Maven where resources may be inside JAR files, producing ajar:file:...!/pathURI thatPaths.get()cannot handle without a ZipFileSystem.The fix replaces
Files.readAllBytes(Paths.get(url.toURI))withurl.openStream().readAllBytes(), which works regardless of whether the resource is a regular file or inside a JAR.Why are the changes needed?
CI runs with Maven and hits
FileSystemNotFoundExceptionwhen initializingTestUDTFJar, causingCliSuiteandHiveThriftBinaryServerSuiteto fail:Reported in #55272 (comment)
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Verified with Maven (the environment where the failure occurs):
Was this patch authored or co-authored using generative AI tooling?
Kiro CLI / Opus 4.6