Skip to content

Commit 9d3d51d

Browse files
surbhigarg92surbhigarg92
authored
chore: Github action for Node spannerlib wrapper (#896)
* chore: Github action for Node spannerlib wrapper * review comments and windows fix * test the release pipeline with PR trigger
1 parent 84d4f75 commit 9d3d51d

7 files changed

Lines changed: 371 additions & 61 deletions

File tree

‎.github/workflows/node-spanner-lib-wrapper-unit-tests.yml‎

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,8 @@ name: Node Wrapper Unit Tests
33
on:
44
push:
55
branches: [ "main" ]
6-
paths:
7-
- 'spannerlib/wrappers/spannerlib-node/**'
86
pull_request:
97
branches: [ "main" ]
10-
paths:
11-
- 'spannerlib/wrappers/spannerlib-node/**'
128
workflow_dispatch:
139

1410
permissions:
@@ -21,25 +17,33 @@ jobs:
2117
strategy:
2218
fail-fast: false
2319
matrix:
24-
os: [ubuntu-latest, macos-latest]
20+
# Use ubuntu-22.04 (glibc 2.35) for Linux to ensure broad binary compatibility.
21+
# Pin to windows-2022 (Visual Studio 2022) to avoid node-gyp auto-detection failure on experimental VS 18 preview on windows-latest.
22+
os: [ubuntu-22.04, macos-latest, windows-2022]
2523
node-version: [22, 24, 26]
2624

25+
env:
26+
npm_config_enable_thin_lto: "false"
27+
npm_config_enable_lto: "false"
28+
npm_config_use_lld: "false"
29+
GYP_DEFINES: "enable_thin_lto=false enable_lto=false use_lld=false lto_jobs="
30+
2731
defaults:
2832
run:
2933
shell: bash
3034
working-directory: ./spannerlib/wrappers/spannerlib-node
3135

3236
steps:
33-
- uses: actions/checkout@v7
37+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
3438

3539
- name: Set up Go
36-
uses: actions/setup-go@v6
40+
uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0
3741
with:
3842
go-version: '1.26.x'
3943
cache-dependency-path: spannerlib/go.sum
4044

4145
- name: Set up Node.js
42-
uses: actions/setup-node@v6
46+
uses: actions/setup-node@1e60f620b9541d16bece96c5465dc8ee9832be0b # v4.0.3
4347
with:
4448
node-version: ${{ matrix.node-version }}
4549
cache: 'npm'
@@ -49,7 +53,9 @@ jobs:
4953
run: npm install
5054

5155
- name: Build Addon and TS
52-
run: npm run build
56+
run: |
57+
unset CFLAGS CXXFLAGS LDFLAGS 2>/dev/null || true
58+
npm run build
5359
5460
- name: Run Unit Tests
5561
run: npm test

‎.github/workflows/release-node-wrapper.yml‎

Lines changed: 204 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
name: Build and Release Node Wrapper
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
platform:
7+
description: 'Target platform package to release'
8+
required: true
9+
type: choice
10+
options:
11+
- 'all'
12+
- 'darwin-arm64'
13+
- 'linux-x64'
14+
- 'linux-arm64'
15+
- 'win32-x64'
16+
default: 'all'
17+
npm_tag:
18+
description: 'NPM distribution tag (e.g. alpha(default), beta, latest, next)'
19+
required: false
20+
type: string
21+
default: 'alpha'
22+
npm_token:
23+
description: 'NPM Token for Wombat (leave empty for dry run)'
24+
required: false
25+
type: string
26+
default: ''
27+
28+
permissions:
29+
contents: read
30+
31+
jobs:
32+
release:
33+
name: Release ${{ matrix.pkg_name }}
34+
runs-on: ${{ matrix.os }}
35+
strategy:
36+
fail-fast: false
37+
matrix:
38+
include:
39+
- os: macos-latest
40+
target: darwin-arm64
41+
os_name: darwin
42+
cpu_name: arm64
43+
goarch: arm64
44+
pkg_name: "@google-cloud/spannerlib-node-darwin-arm64"
45+
- os: ubuntu-22.04
46+
target: linux-x64
47+
os_name: linux
48+
cpu_name: x64
49+
goarch: amd64
50+
pkg_name: "@google-cloud/spannerlib-node-linux-x64"
51+
- os: ubuntu-22.04
52+
target: linux-arm64
53+
os_name: linux
54+
cpu_name: arm64
55+
goarch: arm64
56+
pkg_name: "@google-cloud/spannerlib-node-linux-arm64"
57+
cc: aarch64-linux-gnu-gcc
58+
cxx: aarch64-linux-gnu-g++
59+
ar: aarch64-linux-gnu-ar
60+
- os: windows-2022
61+
target: win32-x64
62+
os_name: win32
63+
cpu_name: x64
64+
goarch: amd64
65+
pkg_name: "@google-cloud/spannerlib-node-win32-x64"
66+
67+
defaults:
68+
run:
69+
shell: bash
70+
working-directory: ./spannerlib/wrappers/spannerlib-node
71+
72+
steps:
73+
- name: Checkout repository
74+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
75+
76+
- name: Check Platform Target
77+
id: check
78+
env:
79+
PLATFORM_INPUT: ${{ inputs.platform || 'all' }}
80+
MATRIX_TARGET: ${{ matrix.target }}
81+
run: |
82+
if [ "$PLATFORM_INPUT" = "all" ] || [ "$PLATFORM_INPUT" = "$MATRIX_TARGET" ]; then
83+
echo "run=true" >> "$GITHUB_OUTPUT"
84+
else
85+
echo "run=false" >> "$GITHUB_OUTPUT"
86+
echo "Skipping matrix target $MATRIX_TARGET (requested: $PLATFORM_INPUT)"
87+
fi
88+
89+
- name: Install ARM64 Cross-Compiler (Linux ARM64)
90+
if: steps.check.outputs.run == 'true' && matrix.target == 'linux-arm64'
91+
run: |
92+
sudo apt-get update
93+
sudo apt-get install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu binutils-aarch64-linux-gnu
94+
95+
- name: Set up Go
96+
if: steps.check.outputs.run == 'true'
97+
uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0
98+
with:
99+
go-version: '1.26.x'
100+
cache-dependency-path: spannerlib/go.sum
101+
102+
- name: Set up Node.js
103+
if: steps.check.outputs.run == 'true'
104+
uses: actions/setup-node@1e60f620b9541d16bece96c5465dc8ee9832be0b # v4.0.3
105+
with:
106+
node-version: '22'
107+
cache: 'npm'
108+
cache-dependency-path: spannerlib/wrappers/spannerlib-node/package.json
109+
110+
- name: Mask NPM Token
111+
if: steps.check.outputs.run == 'true' && inputs.npm_token != ''
112+
env:
113+
NPM_TOKEN: ${{ inputs.npm_token }}
114+
run: |
115+
echo "::add-mask::${NPM_TOKEN}"
116+
117+
- name: Configure NPM registry auth
118+
if: steps.check.outputs.run == 'true' && inputs.npm_token != ''
119+
env:
120+
NPM_TOKEN: ${{ inputs.npm_token }}
121+
run: |
122+
echo "//wombat-dressing-room.appspot.com/:_authToken=${NPM_TOKEN}" > ~/.npmrc
123+
124+
- name: Install dependencies
125+
if: steps.check.outputs.run == 'true'
126+
run: npm install
127+
128+
- name: Build Go library and C++ Addon
129+
if: steps.check.outputs.run == 'true'
130+
env:
131+
npm_config_enable_thin_lto: "false"
132+
npm_config_enable_lto: "false"
133+
npm_config_use_lld: "false"
134+
GYP_DEFINES: "enable_thin_lto=false enable_lto=false use_lld=false lto_jobs="
135+
npm_config_arch: ${{ matrix.cpu_name }}
136+
GOARCH: ${{ matrix.goarch }}
137+
CGO_ENABLED: "1"
138+
run: |
139+
unset CFLAGS CXXFLAGS LDFLAGS 2>/dev/null || true
140+
[ -n "${{ matrix.cc }}" ] && export CC="${{ matrix.cc }}"
141+
[ -n "${{ matrix.cxx }}" ] && export CXX="${{ matrix.cxx }}"
142+
[ -n "${{ matrix.ar }}" ] && export AR="${{ matrix.ar }}"
143+
npm run build
144+
145+
- name: Prepare Platform Package Manifest
146+
if: steps.check.outputs.run == 'true'
147+
env:
148+
PKG_NAME: ${{ matrix.pkg_name }}
149+
OS_NAME: ${{ matrix.os_name }}
150+
CPU_NAME: ${{ matrix.cpu_name }}
151+
run: |
152+
node -e '
153+
const fs = require("fs");
154+
const pkg = JSON.parse(fs.readFileSync("./package.json", "utf8"));
155+
pkg.name = process.env.PKG_NAME;
156+
pkg.os = [process.env.OS_NAME];
157+
pkg.cpu = [process.env.CPU_NAME];
158+
fs.writeFileSync("./package.json", JSON.stringify(pkg, null, 2) + "\n");
159+
console.log("Updated package.json for " + pkg.name + " (" + pkg.os + "/" + pkg.cpu + ")");
160+
'
161+
162+
- name: Package and Publish
163+
if: steps.check.outputs.run == 'true'
164+
env:
165+
NPM_TOKEN: ${{ inputs.npm_token }}
166+
NPM_TAG: ${{ inputs.npm_tag || 'alpha' }}
167+
run: |
168+
set -eo pipefail
169+
export NPM_CONFIG_PREFIX="${HOME}/.npm-global"
170+
171+
# Optional releasetool publisher script reporter if available
172+
if python3 -c "import releasetool" 2>/dev/null; then
173+
python3 -m releasetool publish-reporter-script > /tmp/publisher-script 2>/dev/null || true
174+
if [ -f /tmp/publisher-script ]; then
175+
source /tmp/publisher-script || true
176+
fi
177+
fi
178+
179+
npm pack .
180+
TARBALL=$(ls -1 -t *.tgz | head -1)
181+
echo "Generated tarball: $TARBALL"
182+
183+
# Publish only when a non-empty npm_token is provided
184+
if [ -n "$NPM_TOKEN" ]; then
185+
echo "Publishing $TARBALL with tag '$NPM_TAG' to Wombat registry..."
186+
npm publish --access=public --tag "$NPM_TAG" --registry=https://wombat-dressing-room.appspot.com "$TARBALL"
187+
else
188+
echo "Dry run mode active: No NPM token provided. Skipped publishing $TARBALL."
189+
fi
190+
191+
find node_modules -name package-lock.json -o -name "*.tgz" | xargs rm -f 2>/dev/null || true
192+
193+
- name: Upload Release Tarball Artifact
194+
if: steps.check.outputs.run == 'true'
195+
uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
196+
with:
197+
name: spannerlib-node-${{ matrix.target }}-tarball
198+
path: spannerlib/wrappers/spannerlib-node/*.tgz
199+
200+
- name: Cleanup credentials
201+
if: always()
202+
run: |
203+
rm -f ~/.npmrc
204+
rm -f /tmp/publisher-script 2>/dev/null || true

‎spannerlib/wrappers/spannerlib-node/BUILD_AND_RELEASE.md‎

Lines changed: 48 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -15,25 +15,27 @@ Bridging JavaScript to the native Go SDK involves a sequential compilation pipel
1515
|
1616
(cgo linker bridge)
1717
v
18-
[Go Shared Library]
18+
[Go Static Archive / DLL]
1919
```
2020

2121
## Compilation Phases
2222

23-
### Phase 1: Compiling the Go Shared Library (CGO Link)
23+
### Phase 1: Compiling the Go Library (CGO Link)
2424

2525
Before building the Node.js Addon, the underlying Go codebase must be compiled into an object format that C/C++ can link against.
2626
* **Trigger:** Executed via `npm run build:go` which runs `bash scripts/build-shared-lib.sh`.
27-
* **Action:** The build script invokes the Go compiler with the `-buildmode=c-shared` flag, targeting the primary C-shared entry point located at [shared_lib.go](../../shared/shared_lib.go).
28-
* **Outputs:** Generates a platform-specific native shared library (e.g., `libspannerlib.dylib` on macOS, `.so` on Linux, `.dll` on Windows) along with the corresponding C header file (`libspannerlib.h`). Both files are placed into the `spannerlib/shared/` directory.
27+
* **Action & Outputs:**
28+
* **Linux & macOS (`-buildmode=c-archive`):** Invokes Go with `-buildmode=c-archive` targeting [shared_lib.go](../../shared/shared_lib.go) to generate a static archive (`libspanner.a`) and C header (`libspanner.h`). The static archive embeds all Go runtime and driver symbols directly into the final `spanner_napi.node` binary, eliminating dynamic shared library dependencies, `@loader_path` rpaths, and `.so`/`.dylib` file distribution.
29+
* **Windows (`-buildmode=c-shared`):** Invokes Go with `-buildmode=c-shared` to generate a dynamic shared library (`libspanner.dll`) along with a companion MSVC-compatible import library (`libspanner.lib`) required by MSVC's linker (`link.exe`).
2930

3031
### Phase 2: Compiling the Native Bridge (node-gyp)
3132

32-
Once the Go shared library is generated, the Node.js C++ wrapper is compiled using `node-gyp` to map V8 engine objects into Go pointers.
33+
Once the Go library is generated, the Node.js C++ wrapper is compiled using `node-gyp` to map V8 engine objects into Go pointers.
3334
* **Trigger:** Executed as part of `npm run build` which invokes `node-gyp rebuild`.
34-
* **Action:** Reads the `gyp` build instructions in [binding.gyp](./binding.gyp) to locate the Go header files, and dynamically links the bridge against the generated Go shared object. It compiles the bridge source file [addon.cc](./src/cpp/addon.cc) using the local OS C++ compiler toolchain (e.g., Clang on macOS, GCC on Linux, MSVC on Windows).
35+
* **Action:** Reads the build instructions in [binding.gyp](./binding.gyp) to locate the Go header files, and links the bridge against the generated Go library. It compiles the bridge source file [addon.cc](./src/cpp/addon.cc) using the local OS C++ compiler toolchain (Clang on macOS, GCC on Linux, MSVC on Windows).
36+
* On **Linux and macOS**, `binding.gyp` statically links `libspanner.a` directly into the `.node` binary.
37+
* On **Windows**, `binding.gyp` links against `libspanner.lib` and copies `libspanner.dll` adjacent to the addon in `build/Release/`.
3538
* **Output:** Generates the native Node.js binary file at `build/Release/spanner_napi.node`.
36-
* **Post-build Link Patch (macOS Only):** To ensure portability on macOS without requiring root or global library installs, `npm run postbuild` invokes the `install_name_tool`. This command alters the dynamic linker search path in the `.node` file to use `@loader_path/libspannerlib.dylib`, ensuring Node.js locates the Go dynamic library relatively from the compiled C++ bridge binary path.
3739

3840
### Phase 3: TypeScript Compilation & Dual-Publishing (ESM / CJS)
3941

@@ -51,4 +53,42 @@ To run the entire pipeline end-to-end and generate a fully runnable local build,
5153
```bash
5254
npm run build
5355
```
54-
This builds the underlying Go shared library, links the C++ bridge layer via `node-gyp`, patches dynamic linker paths, and outputs the final dual ESM/CJS JavaScript distributions.
56+
This builds the underlying Go library, links the C++ bridge layer via `node-gyp`, and outputs the final dual ESM/CJS JavaScript distributions.
57+
58+
## Platform-Specific Release Pipelines (GitHub Actions)
59+
60+
Releasing the prebuilt native platform packages is managed via a unified manual GitHub Actions workflow:
61+
* **Workflow:** [release-node-wrapper.yml](../../../.github/workflows/release-node-wrapper.yml)
62+
63+
### Target Platform Packages
64+
65+
| Package Name | Target Platform | Runner OS | glibc / Toolchain |
66+
| :--- | :--- | :--- | :--- |
67+
| **`@google-cloud/spannerlib-node-darwin-arm64`** | macOS (Apple Silicon `arm64`) | `macos-latest` | Apple Clang (`c-archive` static) |
68+
| **`@google-cloud/spannerlib-node-linux-x64`** | Linux (`x64`) | `ubuntu-22.04` | **glibc 2.35** / GCC (`c-archive` static) |
69+
| **`@google-cloud/spannerlib-node-linux-arm64`** | Linux (`arm64`) | `ubuntu-22.04` | **glibc 2.35** / `gcc-aarch64-linux-gnu` (cross-compile) |
70+
| **`@google-cloud/spannerlib-node-win32-x64`** | Windows (`x64`) | `windows-2022` | MSVC 2022 (`c-shared` DLL + `.lib`) |
71+
72+
> **Note on Linux Compatibility:** Compiling on `ubuntu-22.04` dynamically links against **glibc 2.35**, ensuring wide binary compatibility with older and enterprise Linux distributions (such as Debian 12, Ubuntu 22.04+, and RHEL 9). The Linux ARM64 build is cross-compiled on `ubuntu-22.04` using `gcc-aarch64-linux-gnu` rather than running on Ubuntu 24.04 ARM runners to prevent glibc 2.39 lock-in.
73+
74+
### Triggering a Release
75+
The workflow uses `workflow_dispatch` and publishes to the Google Wombat registry (`https://wombat-dressing-room.appspot.com`):
76+
77+
1. Go to the **Actions** tab in GitHub.
78+
2. Select **Build and Release Node Wrapper**.
79+
3. Click **Run workflow**.
80+
4. Provide the inputs:
81+
* **`platform`** *(Required)*: Select `all` (default) to build and release all 4 platforms concurrently, or select a specific target (`darwin-arm64`, `linux-x64`, `linux-arm64`, `win32-x64`).
82+
* **`npm_tag`** *(Optional, default: `alpha`)*: NPM distribution tag (e.g. `alpha`, `beta`, `latest`).
83+
* **`npm_token`** *(Optional)*: The authentication token for `//wombat-dressing-room.appspot.com/:_authToken`. If left empty, the workflow automatically runs in **dry-run mode** (builds, packages, and uploads `.tgz` artifacts without publishing).
84+
85+
### Release Execution Steps
86+
For each target platform in the matrix, the workflow:
87+
1. Sets up Go (`1.26.x`) and Node.js (`22`) environments.
88+
2. Masks the supplied `npm_token` and configures `~/.npmrc` for Wombat registry auth.
89+
3. Installs dependencies (`npm install`).
90+
4. Compiles the Go library (`libspanner.a` / `libspanner.dll`), links the C++ N-API addon, and compiles TypeScript dual outputs (`npm run build`).
91+
5. Sets the target package name (e.g. `@google-cloud/spannerlib-node-linux-x64`) and `os`/`cpu` metadata in `package.json`.
92+
6. Packages the distribution tarball (`npm pack .`) containing both the JavaScript bundles and the compiled native binary (`spanner_napi.node` plus `libspanner.dll` on Windows).
93+
7. Publishes the generated tarball to Wombat with the specified dist-tag (`npm publish --access=public --tag $NPM_TAG --registry=https://wombat-dressing-room.appspot.com`).
94+
8. Archives and uploads the release `.tgz` as a workflow artifact for auditing and SBOM generation.

‎spannerlib/wrappers/spannerlib-node/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Node-API Wrapper for Spanner Shared Library
22

3-
This package provides a high-performance Node-API (N-API) bridge to the Go-based Spanner shared library. It offers superior stability and performance compared to traditional FFI approaches.
3+
> **NOTICE:** This is an internal library intended for use by Google Cloud Spanner driver packages (such as the high-level Node.js Spanner driver). It is not intended for direct use by end customers and can introduce breaking changes without prior notice.
44
55
## Prerequisites
66

0 commit comments

Comments
 (0)