You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix: Keep uni_curl_shim.c buildable on Windows so downstream Native builds link (#640)
**Description**
uni 2026.1.17 broke every downstream Scala Native build on Windows:
```
uni_native0.5_3-2026.1.17-6/scala-native/uni_curl_shim.c:44:10:
fatal error: 'dlfcn.h' file not found
```
The `dlsym` rewrite that fixed#622 reached for `<dlfcn.h>` and
`<pthread.h>` unconditionally, and MSVC ships neither. Because Scala
Native compiles every `.c` under `resources/scala-native/` from every
jar on the classpath, this hits consumers that never touch curl —
[wvlet's Windows native
build](https://github.com/wvlet/wvlet/actions/runs/28837314411) is
failing on `main` for exactly this reason, and it only links `wvc-lib`.
Both Windows jobs there die on this one error and nothing else.
**The fix.** Split the two POSIX primitives per platform, preserving the
property #622 turned on — *the object file must reference no libcurl
symbol*, or downstream links without `-lcurl` fail:
| | POSIX | Windows |
|---|---|---|
| lookup | `dlsym(RTLD_DEFAULT, name)` | `GetProcAddress` over
`EnumProcessModules` |
| run-once | `pthread_once` | `InitOnceExecuteOnce` |
Windows has no all-modules handle, so the shim enumerates the process's
loaded modules and tries each — the process image first, matching
`dlsym`'s order. `#define PSAPI_VERSION 2` keeps `EnumProcessModules` in
`kernel32` (as `K32EnumProcessModules`) rather than `psapi.lib`, since a
jar-resource `.c` can't make downstream binaries add a linker flag. The
POSIX path is unchanged in behaviour.
One consequence worth flagging: runtime lookup only finds *exported*
symbols, so **libcurl must be linked as a DLL**, not a static `.lib`.
vcpkg's default `x64-windows` / `arm64-windows` triplets already do
that. This isn't new to Windows — a static `libcurl.a` is equally
invisible to `dlsym(RTLD_DEFAULT, ...)`. It's the price #622's fix
already paid; the ADR and the user docs now say so out loud.
**Why CI didn't catch it, and what now does.** Two distinct failure
modes, neither check subsuming the other:
1. **`curl shim C (Windows)`** — compiles the shim standalone with clang
on `windows-latest`. Nothing else here compiles the Windows half of the
`#if`.
2. **`check-curl-shim.sh`** — one step in the Linux Native job,
asserting via `nm -u` that the object names no `curl_easy_*` symbol.
*No* Scala Native job can catch that on any OS: [`build.sbt` passes
`-lcurl`
unconditionally](https://github.com/wvlet/uni/blob/main/build.sbt#L101-L103),
so uni's own binaries always resolve those symbols and link happily.
Only a consumer that doesn't link libcurl breaks — which is precisely
how #622 escaped this repo's CI. Inspecting the object stands in for
that consumer.
I checked (2) bites rather than rubber-stamps: against `v2026.1.16`'s
`extern`-based shim it reports `_curl_easy_setopt` /
`_curl_easy_getinfo` and exits 1, reproducing #622.
Both are gated on a new narrow `native` paths-filter (`**.c`,
`**/.native/**`, `project/**`, the workflow itself), so they run on
every push to `main` and only on PRs that can actually break them.
(`dorny/paths-filter` matches with `dot: true`, so `**/.native/**` does
reach the dot-prefixed source trees.) The `changes` filter never watched
`**.c` at all before, so an edit touching only this file skipped CI
outright.
> [!IMPORTANT]
> **A real `Scala Native (Windows)` job would be the better guard, and
uni cannot run one.** I built it — LLVM, vcpkg libcurl/zlib/OpenSSL,
aliasing every `foo.lib` that `-lfoo` asks for — and it compiles
everything, then fails at the final link:
>
> ```
> error LNK2019: unresolved external symbol scalanative_pollin
> referenced in function ...wvlet.uni.http.NativeServerTest...
> fatal error LNK1120: 5 unresolved externals
> ```
>
> `NativeServer` uses POSIX `poll()`, and Scala Native's
[`posixlib/poll.c`](https://github.com/scala-native/scala-native/blob/v0.5.12/posixlib/src/main/resources/scala-native/poll.c)
is wrapped in `#if defined(__unix__) || (defined(__APPLE__) &&
defined(__MACH__))` — those symbols don't exist on Windows. Every native
HTTP test starts a server, so uni's native test binary is unlinkable
there, full stop. Giving `NativeServer` a `WSAPoll` path would unblock
it; this PR's history has the working toolchain setup to reuse. Worth
filing as a follow-up issue.
>
> That discarded run wasn't wasted: it linked everything *except*
`poll`, so none of the unresolved symbols were `curl_easy_*` — proving
the shim compiles **and links** clean under clang/MSVC, which is the
regression this PR is about.
**Docs.** The old text said only *"Scala Native requires libcurl to be
available at runtime."* Added [Linking libcurl on Scala
Native](https://github.com/wvlet/uni/blob/fix/curl-shim-windows/docs/http/client.md#linking-libcurl-on-scala-native)
covering the three things a Native user actually hits: you need libcurl
*only* if you use the HTTP client (DCE drops `@link("curl")` otherwise —
that's what #622 bought, and nobody would guess it); it must be a shared
library; and on Windows you alias `libcurl.lib` to the `curl.lib` Scala
Native asks for.
**Related Issue/Task**
Follow-up to #622 / the ADR added in #631. Unblocks wvlet's `Native`
workflow on `main`.
**Checklist**
- [x] This pull request focuses on a single task.
- [x] The change does not contain security credentials
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Copy file name to clipboardExpand all lines: adr/README.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,4 +12,4 @@ Read the relevant ADR before modifying the area it covers. Add a new entry here
12
12
-[`2026-06-30-sbt-uni-crossproject.md`](2026-06-30-sbt-uni-crossproject.md) — the `sbt-uni-crossproject/` build: a minimal, uni-owned sbt 2.x re-implementation of `portable-scala/sbt-crossproject` (which isn't ported to sbt 2.x), supporting only the `CrossType.Pure` layout uni uses. Read before touching that build; covers the single-plugin-for-all-three-platforms choice, the Scala 3 val-name macro replicating sbt's `KeyMacro.definingValName`, why internal materialization needs `new CrossProject(...)`, and the `given Conversion[Builder, CrossProject]` build trigger.
13
13
-[`2026-06-30-sbt2-main-build-migration.md`](2026-06-30-sbt2-main-build-migration.md) — migrating the **main build** to sbt 2.x: swaps the unported third-party plugins for the uni-owned ones (`sbt-uni-crossproject`, `uni-jsenv-playwright`, `sbt-uni` for `sbt-revolver`). Read before touching `build.sbt` / `project/plugin.sbt`; covers the output-dir name collision (root → `uni-root`), `%%%`→`%%` and the `scalajs-test-interface_2.13` single-`%` exception, `Def.uncached` for `jsEnv`, and the `implicitConversions` import.
14
14
-[`2026-07-06-plugin-extension-points.md`](2026-07-06-plugin-extension-points.md) — `wvlet.uni.plugin` is built on typed `ExtensionPoint`s (identity-compared singletons; keyed points reject duplicate ids at activation), with `PluginContext` reduced to `contribute` + `onDeactivate`. Read before adding new contribution kinds: define a point next to the contributed type (`Command.point`, `RPCPlugin.routerPoint`) so dependency arrows point into `plugin`, never out of it.
15
-
-[`2026-07-06-curl-shim-weak-linking.md`](2026-07-06-curl-shim-weak-linking.md) — `uni_curl_shim.c` resolves libcurl's variadic setopt/getinfo via `dlsym(RTLD_DEFAULT, ...)` inside a `pthread_once` init, not C-level `extern`s, so downstream Scala Native builds that don't use `CurlBindings` still link (issue #622). Read before touching the shim: covers why `__attribute__((weak))` on the extern fails on macOS, why `RTLD_DEFAULT` (not `dlopen`) is the right lookup, why the variadic-typedef ABI trick from #580 is still load-bearing, why `_GNU_SOURCE` is required on glibc, and why lazy init needs `pthread_once` not just "same value written twice".
15
+
- [`2026-07-06-curl-shim-weak-linking.md`](2026-07-06-curl-shim-weak-linking.md) — `uni_curl_shim.c` resolves libcurl's variadic setopt/getinfo from the process's already-loaded modules (`dlsym(RTLD_DEFAULT, ...)` on POSIX, `GetProcAddress` over `EnumProcessModules` on Windows) inside a run-once init, not C-level `extern`s, so downstream Scala Native builds that don't use `CurlBindings` still link (issue #622). Read before touching the shim: covers why `__attribute__((weak))` on the extern fails on macOS, why `RTLD_DEFAULT` (not `dlopen`) is the right lookup, why MSVC needs the whole `#if defined(_WIN32)` half (no `<dlfcn.h>`/`<pthread.h>`) and `PSAPI_VERSION 2`, why libcurl must now be linked dynamically, why the variadic-typedef ABI trick from #580 is still load-bearing, why `_GNU_SOURCE` is required on glibc, why lazy init needs a run-once guard not just "same value written twice", why a real Scala Native Windows CI job is impossible today (`NativeServer` needs POSIX `poll`), and why guarding this takes a standalone clang compile plus a separate `nm` check (the build's unconditional `-lcurl` blinds every native job to the #622 regression).
0 commit comments