fix: Resolve libcurl symbols via dlsym so Scala Native binaries without CurlBindings link (#622) - #636
Conversation
There was a problem hiding this comment.
Code Review
This pull request addresses a linking issue in downstream Scala Native projects (issue #622) by modifying uni_curl_shim.c to dynamically resolve libcurl symbols (curl_easy_setopt and curl_easy_getinfo) via dlsym instead of using static extern declarations, and documents this change in a new Architecture Decision Record (ADR). The review feedback points out that the lazy initialization of the function pointers is not thread-safe and could lead to undefined behavior, suggesting pthread_once as a solution. Additionally, defining _GNU_SOURCE is recommended to ensure the portability of RTLD_DEFAULT on Linux systems.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| static uni_curl_setopt_fn uni_curl_setopt_p = NULL; | ||
| static uni_curl_getinfo_fn uni_curl_getinfo_p = NULL; | ||
|
|
||
| static void *uni_curl_shim_resolve(const char *name) { | ||
| void *sym = dlsym(RTLD_DEFAULT, name); | ||
| if (sym == NULL) { | ||
| fprintf( | ||
| stderr, | ||
| "uni_curl_shim: libcurl symbol '%s' not found in this process. Add " | ||
| "wvlet.uni.http.CurlBindings to your Scala Native build (which links " | ||
| "libcurl via @link(\"curl\")), or install libcurl. See wvlet/uni#622.\n", | ||
| name); | ||
| abort(); | ||
| } | ||
| return sym; | ||
| } | ||
|
|
||
| int uni_curl_easy_setopt_ptr(void *handle, int option, void *value) { | ||
| return curl_easy_setopt(handle, option, value); | ||
| if (uni_curl_setopt_p == NULL) | ||
| uni_curl_setopt_p = (uni_curl_setopt_fn)uni_curl_shim_resolve("curl_easy_setopt"); | ||
| return uni_curl_setopt_p(handle, option, value); | ||
| } | ||
|
|
||
| int uni_curl_easy_setopt_long(void *handle, int option, long value) { | ||
| return curl_easy_setopt(handle, option, value); | ||
| if (uni_curl_setopt_p == NULL) | ||
| uni_curl_setopt_p = (uni_curl_setopt_fn)uni_curl_shim_resolve("curl_easy_setopt"); | ||
| return uni_curl_setopt_p(handle, option, value); | ||
| } | ||
|
|
||
| /* Typed long* out-parameter for CURLINFO_*_RESPONSE_CODE and other `long` infos. */ | ||
| int uni_curl_easy_getinfo_long(void *handle, int info, long *value) { | ||
| return curl_easy_getinfo(handle, info, value); | ||
| if (uni_curl_getinfo_p == NULL) | ||
| uni_curl_getinfo_p = (uni_curl_getinfo_fn)uni_curl_shim_resolve("curl_easy_getinfo"); | ||
| return uni_curl_getinfo_p(handle, info, value); | ||
| } |
There was a problem hiding this comment.
The current lazy initialization of uni_curl_setopt_p and uni_curl_getinfo_p is not thread-safe. If multiple threads concurrently invoke the shim functions when the pointers are NULL, they will concurrently read and write to these static variables. In C11, concurrent unsynchronized accesses to non-atomic variables where at least one is a write constitutes a data race, which is undefined behavior (UB). Even if they write the same value, compiler optimizations can lead to unexpected behavior.
To resolve this portably and efficiently without adding complex locks or relying on compiler-specific atomics, use POSIX's pthread_once. This guarantees thread-safe, one-time initialization of both function pointers.
static uni_curl_setopt_fn uni_curl_setopt_p = NULL;
static uni_curl_getinfo_fn uni_curl_getinfo_p = NULL;
static pthread_once_t uni_curl_init_once = PTHREAD_ONCE_INIT;
static void *uni_curl_shim_resolve(const char *name) {
void *sym = dlsym(RTLD_DEFAULT, name);
if (sym == NULL) {
fprintf(
stderr,
"uni_curl_shim: libcurl symbol '%s' not found in this process. Add "
"wvlet.uni.http.CurlBindings to your Scala Native build (which links "
"libcurl via @link(\\\"curl\\\")), or install libcurl. See wvlet/uni#622.\\n",
name);
abort();
}
return sym;
}
static void uni_curl_shim_init(void) {
uni_curl_setopt_p = (uni_curl_setopt_fn)uni_curl_shim_resolve("curl_easy_setopt");
uni_curl_getinfo_p = (uni_curl_getinfo_fn)uni_curl_shim_resolve("curl_easy_getinfo");
}
int uni_curl_easy_setopt_ptr(void *handle, int option, void *value) {
pthread_once(&uni_curl_init_once, uni_curl_shim_init);
return uni_curl_setopt_p(handle, option, value);
}
int uni_curl_easy_setopt_long(void *handle, int option, long value) {
pthread_once(&uni_curl_init_once, uni_curl_shim_init);
return uni_curl_setopt_p(handle, option, value);
}
/* Typed long* out-parameter for CURLINFO_*_RESPONSE_CODE and other `long` infos. */
int uni_curl_easy_getinfo_long(void *handle, int info, long *value) {
pthread_once(&uni_curl_init_once, uni_curl_shim_init);
return uni_curl_getinfo_p(handle, info, value);
}There was a problem hiding this comment.
Addressed in the rebased commit — the shim now uses pthread_once around a single uni_curl_shim_init that resolves both function pointers. Thanks for the catch: my earlier "same value written twice" reasoning was wrong per C11's data-race rules, and the ADR is updated to say so.
| #include <dlfcn.h> | ||
| #include <stdio.h> | ||
| #include <stdlib.h> |
There was a problem hiding this comment.
On Linux systems using glibc, the RTLD_DEFAULT constant is a GNU extension and is only exposed in <dlfcn.h> if the _GNU_SOURCE feature test macro is defined before including the header. To ensure portable compilation across all Linux distributions and toolchains, define _GNU_SOURCE at the very top of the file.
#define _GNU_SOURCE
#include <dlfcn.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>There was a problem hiding this comment.
Addressed — #define _GNU_SOURCE is now the first line of the file, before <dlfcn.h>. On macOS libSystem defines RTLD_DEFAULT unconditionally so the macro is harmless there.
…tive binaries that don't use CurlBindings still link (#622) Since v2026.1.13, Scala Native builds that depended on uni-native failed to link with `undefined reference to curl_easy_setopt` whenever the downstream code did not touch `CurlBindings` — Scala Native compiles `resources/scala- native/*.c` unconditionally, and the shim's strong `extern` references to libcurl symbols could not be satisfied because `-lcurl` only reaches the linker via `@link("curl")` on `CurlBindings.Extern`, which DCE strips. Resolve `curl_easy_setopt`/`curl_easy_getinfo` lazily via `dlsym(RTLD_DEFAULT, ...)` instead of declaring them as C-level `extern`s, so the compiled `.o` carries no libcurl symbol references. Variadic ABI (the whole reason the shim exists — #580) is preserved by declaring the function-pointer typedefs variadic. Weak-`extern` was tried first but is unusable — macOS `ld` still requires `-undefined dynamic_lookup` to leave weak-undefineds unresolved at final link, and a jar-resource `.c` cannot force that flag on downstream builds. See adr/2026-07-06-curl-shim-weak-linking.md.
7a4d67a to
34667cb
Compare
Summary
Fixes #622. Since v2026.1.13, Scala Native builds depending on
uni-nativefail to link withundefined reference to curl_easy_setoptwhenever downstream code doesn't touchCurlBindings(e.g. a project that only usesLogSupport). Scala Native compilesresources/scala-native/*.cfrom every jar on the classpath unconditionally; the shim's strongexternreferences to libcurl symbols couldn't be satisfied because-lcurlonly reaches the linker via@link("curl")onCurlBindings.Extern, which DCE strips when unused.uni_curl_shim.cnow resolvescurl_easy_setopt/curl_easy_getinfolazily viadlsym(RTLD_DEFAULT, ...)inside apthread_onceinit, instead of C-levelexterns. The compiled.ocarries no libcurl symbol references, so downstream links succeed regardless of whether libcurl is on the linker command line.#define _GNU_SOURCEbefore<dlfcn.h>soRTLD_DEFAULTis exposed on glibc; harmless on macOS.__attribute__((weak))on the extern was tried first — works on Linux, fails on macOS becauseldstill needs-undefined dynamic_lookupfor weak-undefineds, and a jar-resource.ccannot force that flag on downstream builds.Design decision documented in
adr/2026-07-06-curl-shim-weak-linking.md.Test plan
clang -c uni_curl_shim.c— the resulting.ohas zero libcurl symbol references (nm -mshows only_abort,_dlsym,_fprintf,_pthread_once,___stderrpundefined)..o— succeeds (simulates the ScalaNative - unable to build when using uni::2026.1.13 (works with uni::2026.1.12 and earlier) #622 downstream scenario)..o(fromtarget/out/native0.5/.../uni_curl_shim.c.o) also shows no libcurl externals../sbt "uniNative/testOnly *NativeCurlClientTest"— 3/3 pass (variadic ABI still correct end-to-end)../sbt "projectNative/test"— 327/327 pass (no regressions).scala-cli run . --nativecase — to be validated post-release.