Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build wheels for Linux and Windows on Github actions #38

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions .github/workflows/pythonpackage.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
name: Python package

on:
push:
release:
types:
- published

jobs:
build_wheels_linux:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
submodules: 'true'
- name: Build wheels
uses: pypa/[email protected]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks reasonable: seems to be a maintained and active project.

env:
CIBW_BEFORE_ALL: "sh linux-build.sh"
CIBW_BEFORE_BUILD: echo {project}/binaries
CIBW_SKIP: "cp36-* pp*"
CIBW_ARCHS_LINUX: auto64
CIBW_REPAIR_WHEEL_COMMAND: "LD_LIBRARY_PATH=/project/binaries auditwheel repair -w {dest_dir} {wheel}"
- uses: actions/upload-artifact@v3
with:
path: ./wheelhouse/*.whl

build_wheels_windows:
runs-on: windows-latest
strategy:
matrix:
arch:
- AMD64
- x86
steps:
- uses: actions/checkout@v3
with:
submodules: 'true'
- uses: actions/setup-python@v4
with:
python-version: '3.8'
- uses: ilammy/msvc-dev-cmd@v1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one I will need to dig into a little bit, to see how legit it is...

Is it recommended by the CIBW folks?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks legit to me, although no mention of it in the CIBW docs that I can see.

I don't know much about Windows; is this needed so that get-nasm.py script works?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It’s necessary to setup the environment for nmake.

with:
arch: ${{ matrix.arch }}
- name: Build dependencies
shell: cmd
run: |
python get-nasm.py ${{ matrix.arch }}
set path=%path%;%CD%\nasm-2.14.02
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this 2.14.02 string should be an argument to the get-nasm.py script? Easier to maintain (?)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong opinion on this. If you want I can change it.

mkdir binaries
cd binaries
cmake -G"NMake Makefiles" -DCMAKE_BUILD_TYPE=Release ../libjpeg-turbo
nmake
cd ..
- name: Build wheels
uses: pypa/[email protected]
env:
CIBW_SKIP: cp36-* pp*
CIBW_ARCHS_WINDOWS: ${{ matrix.arch }}
CIBW_BEFORE_BUILD_WINDOWS: "python -m pip install delvewheel"
CIBW_REPAIR_WHEEL_COMMAND_WINDOWS: "python -m delvewheel repair -w {dest_dir} {wheel} --add-path binaries"
- uses: actions/upload-artifact@v3
with:
path: ./wheelhouse/*.whl


upload-pypi:
if: github.event_name == 'release' && github.event.action == 'published'
needs: [build_wheels_linux, build_wheels_windows]
runs-on: ubuntu-latest

steps:
- uses: actions/download-artifact@v2
with:
name: artifact
path: dist

- uses: pypa/gh-action-pypi-publish@master
with:
user: __token__
password: ${{ secrets.pypi_password }}
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
*.pyc
__pycache__
doc/_*

build/
binaries/
CMakeFiles/
4 changes: 4 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[submodule "libjpeg-turbo"]
path = libjpeg-turbo
url = https://github.com/libjpeg-turbo/libjpeg-turbo
branch = main
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Wand and PIL were too slow to be usable.
.. _PIL/PIllow: https://pillow.readthedocs.io
.. _Benchmarks: https://jpegtran-cffi.readthedocs.io/en/latest/#benchmarks
.. _epeg library: https://github.com/mattes/epeg
.. _libturbojpeg: http://www.libjpeg-turbo.org/About/TurboJPEG
.. _turbojpeg: http://www.libjpeg-turbo.org/About/TurboJPEG
.. _libjpeg-turbo: http://www.libjpeg-turbo.org/
.. _CFFI: https://cffi.readthedocs.io
.. _spreads: https://spreads.readthedocs.io
Expand Down
28 changes: 28 additions & 0 deletions get-nasm.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import urllib.request
from argparse import ArgumentParser
from pathlib import Path
from zipfile import ZipFile

urls = {
"amd64": "https://www.nasm.us/pub/nasm/releasebuilds/2.14.02/win64/nasm-2.14.02-win64.zip",
"x86": "https://www.nasm.us/pub/nasm/releasebuilds/2.14.02/win32/nasm-2.14.02-win32.zip",
}


def main():
parser = ArgumentParser()
parser.add_argument("arch", type=str.lower, choices=urls.keys())
parser.add_argument("--zip-name", default="nasm.zip")
args = parser.parse_args()

url = urls[args.arch]
urllib.request.urlretrieve(url, args.zip_name)

with ZipFile(args.zip_name, "r") as zip:
zip.extractall()

assert Path("nasm-2.14.02").exists()


if __name__ == "__main__":
main()
8 changes: 6 additions & 2 deletions jpegtran/jpegtran_build.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,17 @@
#include "turbojpeg.h"
"""

static_lib_dir = 'binaries'

ffi = FFI()
ffi.set_source(
"_jpegtran", SOURCE,
sources=["src/epeg.c"],
include_dirs=["src"],
include_dirs=["src", "libjpeg-turbo", static_lib_dir],
define_macros=[("HAVE_UNSIGNED_CHAR", "1")],
libraries=["jpeg", "turbojpeg"])
libraries=['jpeg', 'turbojpeg'],
library_dirs=[static_lib_dir],
)
ffi.cdef(CDEF)

if __name__ == "__main__":
Expand Down
1 change: 1 addition & 0 deletions libjpeg-turbo
Submodule libjpeg-turbo added at 3a5362
10 changes: 10 additions & 0 deletions linux-build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
yum makecache
yum -y install nasm
python -m pip install -U pip wheel
python -m pip install -U cmake
mkdir binaries
cd binaries
cmake -G"Unix Makefiles" ../libjpeg-turbo
make
cd ..
ls binaries
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[build-system]
requires = ["setuptools", "cffi >= 1.0"]
build-backend = "setuptools.build_meta"
11 changes: 5 additions & 6 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@

if os.path.exists('README.rst'):
if sys.version_info > (3,):
description_long = open('README.rst', encoding="utf-8").read()
long_description = open('README.rst', encoding="utf-8").read()
else:
description_long = open('README.rst').read()
long_description = open('README.rst').read()
else:
description_long = """
long_description = """
A Python package for blazingly fast JPEG transformations. Compared to other,
more general purpose image processing libraries like `wand-py`_ or
`PIL/Pillow`_, the performance gain can, depending on the transformation, be
Expand All @@ -20,17 +20,16 @@
"""

setup(
name='jpegtran-cffi',
name='jpegtran-cffi-wheels',
version="0.5.3.dev",
description=("Extremly fast, (mostly) lossless JPEG transformations"),
description_long=description_long,
long_description=long_description,
author="Johannes Baiter",
url="http://github.com/jbaiter/jpegtran-cffi.git",
author_email="[email protected]",
license='MIT',
packages=['jpegtran'],
package_data={'jpegtran': ['jpegtran.cdef']},
setup_requires=['cffi >= 1.0'],
install_requires=['cffi >= 1.0'],
cffi_modules=["jpegtran/jpegtran_build.py:ffi"]
)