Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 24 additions & 32 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,44 +32,36 @@ jobs:
- name: Build
run: python -m build --sdist

- uses: actions/upload-artifact@v3
- uses: actions/upload-artifact@v4
with:
name: sdist
path: dist/*
if-no-files-found: error

# We build for Linux using uraimo/run-on-arch-action@v2, which runs a container under
# the runner in order to reach different platforms (notably Alpine with its musl) and
# architectures (notably aarch64) via emulation. uraimo/run-on-arch-action@v2 doesn't
# support Mac or Windows, so we run a separate job for those.
# architectures. uraimo/run-on-arch-action@v2 doesn't support Mac or Windows, so we
# run a separate job for those.
linux-wheels:
name: Build wheel for ${{ matrix.image }}
runs-on: ubuntu-latest
name: Build wheel for ${{ matrix.config.image }} on ${{ matrix.config.os }}
runs-on: ${{ matrix.config.os }}
strategy:
fail-fast: false
matrix:
image:
config:
# To maximize compatibility of generated wheels, we should theoreticaly build
# on the *oldest* supported distros.
# But V8 ships its own bullseye sysroot and links against that, so we may as
# well run on bullseye (even though buster would provide an older supported
# build distro):
- debian:bullseye
- arm64v8/debian:bullseye
# Alpine 3.19 includes a clang new enough for V8 to build (with only minor
# patches!). Builds on 3.19 seem incompatible with <= 3.18 due to libstdc++
# symbols. (And we can't just run on an old Alpine and update clang from the
# llvm site, because unlike Debian, the llvm project doesn't maintain
# updated packages for old Alpine distros.)
- alpine:3.19
- arm64v8/alpine:3.19
exclude:
# The aarch64 builds run really slowly, so let's skip them except for
# releases.
# For more info on this GitHub Actions hack, see:
# https://stackoverflow.com/questions/65384420/how-do-i-make-a-github-action-matrix-element-conditional#answer-73822998
- image: ${{ !startsWith(github.ref, 'refs/heads/release/') && 'arm64v8/debian:bullseye' }}
- image: ${{ !startsWith(github.ref, 'refs/heads/release/') && 'arm64v8/alpine:3.19' }}
- os: ubuntu-latest # x86_64 base OS (on which we run a container)
image: debian:bullseye
- os: ubuntu-latest
image: alpine:3.19
- os: ubuntu-24.04-arm
image: arm64v8/debian:bullseye
- os: ubuntu-24.04-arm
image: arm64v8/alpine:3.19

steps:
- name: Configure git
Expand All @@ -90,7 +82,7 @@ jobs:
with:
arch: none
distro: none
base_image: ${{ matrix.image }}
base_image: ${{ matrix.config.image }}

setup: |
mkdir -p "${PWD}/wheels"
Expand All @@ -101,7 +93,7 @@ jobs:
shell: /bin/sh

install: |
case "${{ matrix.image }}" in
case "${{ matrix.config.image }}" in
*debian*)
# Let's download some system packages!
# Note that the precise list of packages we need is intertwined not just
Expand Down Expand Up @@ -232,7 +224,7 @@ jobs:
cp dist/*.whl /wheels/
chmod a+rwx /wheels/*.whl

- uses: actions/upload-artifact@v3
- uses: actions/upload-artifact@v4
with:
name: wheels
path: ./wheels/*
Expand All @@ -243,7 +235,7 @@ jobs:
with:
arch: none
distro: none
base_image: ${{ matrix.image }}
base_image: ${{ matrix.config.image }}

setup: |
mkdir -p "${PWD}/wheels"
Expand All @@ -254,7 +246,7 @@ jobs:
shell: /bin/sh

install: |
case "${{ matrix.image }}" in
case "${{ matrix.config.image }}" in
*debian*)
PACKAGES=""
PACKAGES="${PACKAGES} python3"
Expand All @@ -280,7 +272,7 @@ jobs:

python3 -m venv /venv
. /venv/bin/activate
case "${{ matrix.image }}" in
case "${{ matrix.config.image }}" in
*debian*)
python3 -m pip install --upgrade hatch hatch-fancy-pypi-readme hatch-mkdocs
hatch run testinstalled:install /wheels/*.whl
Expand All @@ -307,7 +299,7 @@ jobs:
# Note that to maximize compatibility of generated wheels, we build on the
# *oldest* supported GitHub-hosted runners, per
# https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners
- os: windows-2019 # x86_64
- os: windows-latest # x86_64
- os: macos-11 # this is the earliest x86_64 runner
- os: macos-14 # this is the earliest arm64 runner

Expand Down Expand Up @@ -336,7 +328,7 @@ jobs:
python3 -m pip install --upgrade build
python3 -m build --wheel

- uses: actions/upload-artifact@v3
- uses: actions/upload-artifact@v4
with:
name: wheels
path: dist/*
Expand All @@ -359,7 +351,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/download-artifact@v3
- uses: actions/download-artifact@v4

- name: Compute release version
run: |
Expand All @@ -386,7 +378,7 @@ jobs:
permissions:
id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
steps:
- uses: actions/download-artifact@v3
- uses: actions/download-artifact@v4

- name: Make dist directory
run: mkdir dist && cp wheels/* sdist/* dist/
Expand Down
9 changes: 7 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,8 +154,8 @@ PyMiniRacer!) which wants to integrate V8 must first build it.
### Build PyPI wheels

Because V8 takes so long to build (about 2-3 hours at present on the free GitHub Actions
runners, and >12 hours when emulating `aarch64` on them), we want to build wheels for
PyPI. We don't want folks to have to build V8 when they `pip install mini-racer`!
runners), we want to build wheels for PyPI. We don't want folks to have to build V8 when
they `pip install mini-racer`!

We build wheels for many operating systems and architectures based on popular demand via
GitHib issues. Currently the list is
Expand Down Expand Up @@ -238,6 +238,11 @@ wouldn't simplify the overall workflow management.

### Use `sccache` to patch around build timeouts

**Update: We still use `sscache` but with
[the new Github-hosted `aarch64` runners](https://github.com/actions/runner-images/issues/10820),
this should no longer be strictly required. We no longer run builds on emulation as
discussed below.**

As of this writing, the Linux `aarch64` builds run on emulation becaues GitHub Actions
has no free hosted `aarch64` runners for Linux. This makes them so slow, they struggle
to complete at all. They take about 24 hours to run. The GitHub Actions
Expand Down
2 changes: 2 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ If you are proposing a feature:
Ready to contribute? Here's how to set up `PyMiniRacer` for local development.

!!! warning

Building this package from source takes several GB of disk space and takes 1-2 hours.

1. Do a quick scan through [the architecture guide](ARCHITECTURE.md) before diving in.
Expand Down Expand Up @@ -220,6 +221,7 @@ To make an ordinary release from `main`:
wheels to PyPI automatically.
!!! warning
As of this writing, the `aarch64` Linux builds are slow because they're running on
emulation. They time out on the first try (and second and third and...) after 6
hours. If you "restart failed jobs", they will quickly catch up to where where they
Expand Down
2 changes: 1 addition & 1 deletion HISTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@

## 0.1.9 (2017-03-24)

- Fix the compilation for Ubuntu 12.04 and glibc \< 2.17.
- Fix the compilation for Ubuntu 12.04 and glibc < 2.17.

## 0.1.8 (2017-03-02)

Expand Down
2 changes: 1 addition & 1 deletion helpers/v8_build.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
LOGGER = getLogger(__name__)
LOGGER.setLevel(DEBUG)
ROOT_DIR = dirname(abspath(__file__))
V8_VERSION = "branch-heads/12.6"
V8_VERSION = "branch-heads/13.2"


def local_path(path="."):
Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ dependencies = [
]

[[tool.hatch.envs.test.matrix]]
python = ["3.8", "3.9", "3.10", "3.11", "3.12"]
python = ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]

# An environment to test installed wheels
[tool.hatch.envs.testinstalled]
Expand All @@ -120,7 +120,7 @@ dependencies = [
]

[[tool.hatch.envs.testinstalled.matrix]]
python = ["3.8", "3.9", "3.10", "3.11", "3.12"]
python = ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]

[tool.hatch.envs.testinstalled.scripts]
# A weird trick to smuggle the just-built wheel into the env, for matrix testing:
Expand Down
10 changes: 7 additions & 3 deletions src/py_mini_racer/_value_handle.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
from datetime import datetime, timezone
from typing import (
TYPE_CHECKING,
Any,
ClassVar,
)

Expand All @@ -23,9 +24,12 @@
PythonJSConvertedTypes,
)

if TYPE_CHECKING:
from collections.abc import Sequence


class _RawValueUnion(ctypes.Union):
_fields_: ClassVar[list[tuple[str, object]]] = [
_fields_: ClassVar[Sequence[tuple[str, Any]]] = [
("value_ptr", ctypes.c_void_p),
("bytes_val", ctypes.POINTER(ctypes.c_char)),
("char_p_val", ctypes.c_char_p),
Expand All @@ -35,7 +39,7 @@ class _RawValueUnion(ctypes.Union):


class _RawValue(ctypes.Structure):
_fields_: ClassVar[list[tuple[str, object]]] = [
_fields_: ClassVar[Sequence[tuple[str, Any]]] = [
("value", _RawValueUnion),
("len", ctypes.c_size_t),
("type", ctypes.c_uint8),
Expand All @@ -52,7 +56,7 @@ class _RawValue(ctypes.Structure):
class _ArrayBufferByte(ctypes.Structure):
# Cannot use c_ubyte directly because it uses <B
# as an internal type but we need B for memoryview.
_fields_: ClassVar[list[tuple[str, object]]] = [
_fields_: ClassVar[list[tuple[str, Any]]] = [
("b", ctypes.c_ubyte),
]
_pack_ = 1
Expand Down
Loading