Description
Starting from tiktoken==0.12.0, the release no longer provides precompiled manylinux2014_x86_64 wheels, only newer manylinux_2_28 (Amazon Linux 2023) and musllinux variants.
This change breaks CI/CD pipelines and AWS Lambda build environments still based on Amazon Linux 2 (glibc 2.26/2.27, manylinux_2_17), where pip must compile from source — which fails because no Rust compiler is available.
This issue affects not only direct users of tiktoken, but also indirect dependents such as:
Both of these packages declare a dependency range like:
As a result, any build that installs these libraries will automatically pull the latest tiktoken (currently 0.12.0) — causing failures in CI environments or Docker images based on Amazon Linux 2, such as the official AWS SAM build images.
This makes otherwise functional codebases fail during dependency resolution, even though tiktoken==0.11.0 worked correctly.
Steps to Reproduce:
- Use the AWS SAM build image:
public.ecr.aws/sam/build-python3.11:latest
(Amazon Linux 2 / manylinux_2_17 compatible)
- Start container and then Inside run:
❯ docker run --platform linux/amd64 --rm -it public.ecr.aws/sam/build-python3.11:latest
bash-4.2# pip install tiktoken
Collecting tiktoken
Downloading tiktoken-0.12.0.tar.gz (37 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting regex>=2022.1.18 (from tiktoken)
Downloading regex-2025.9.18-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (40 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.5/40.5 kB 10.2 MB/s eta 0:00:00
Collecting requests>=2.26.0 (from tiktoken)
Downloading requests-2.32.5-py3-none-any.whl.metadata (4.9 kB)
Collecting charset_normalizer<4,>=2 (from requests>=2.26.0->tiktoken)
Downloading charset_normalizer-3.4.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (36 kB)
Collecting idna<4,>=2.5 (from requests>=2.26.0->tiktoken)
Downloading idna-3.10-py3-none-any.whl.metadata (10 kB)
Requirement already satisfied: urllib3<3,>=1.21.1 in /var/lang/lib/python3.11/site-packages (from requests>=2.26.0->tiktoken) (1.26.19)
Collecting certifi>=2017.4.17 (from requests>=2.26.0->tiktoken)
Downloading certifi-2025.10.5-py3-none-any.whl.metadata (2.5 kB)
Downloading regex-2025.9.18-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (798 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 799.0/799.0 kB 9.1 MB/s eta 0:00:00
Downloading requests-2.32.5-py3-none-any.whl (64 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.7/64.7 kB 14.0 MB/s eta 0:00:00
Downloading certifi-2025.10.5-py3-none-any.whl (163 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 163.3/163.3 kB 10.3 MB/s eta 0:00:00
Downloading charset_normalizer-3.4.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (150 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 150.3/150.3 kB 9.4 MB/s eta 0:00:00
Downloading idna-3.10-py3-none-any.whl (70 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.4/70.4 kB 12.2 MB/s eta 0:00:00
Building wheels for collected packages: tiktoken
Building wheel for tiktoken (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for tiktoken (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [50 lines of output]
/tmp/pip-build-env-v2vl7fwu/overlay/lib/python3.11/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated
!!
********************************************************************************
Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0).
By 2026-Feb-18, you need to update your project and remove deprecated calls
or your builds will no longer be supported.
See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
********************************************************************************
!!
corresp(dist, value, root_dir)
running bdist_wheel
running build
running build_py
creating build/lib.linux-x86_64-cpython-311/tiktoken
copying tiktoken/_educational.py -> build/lib.linux-x86_64-cpython-311/tiktoken
copying tiktoken/model.py -> build/lib.linux-x86_64-cpython-311/tiktoken
copying tiktoken/registry.py -> build/lib.linux-x86_64-cpython-311/tiktoken
copying tiktoken/core.py -> build/lib.linux-x86_64-cpython-311/tiktoken
copying tiktoken/__init__.py -> build/lib.linux-x86_64-cpython-311/tiktoken
copying tiktoken/load.py -> build/lib.linux-x86_64-cpython-311/tiktoken
creating build/lib.linux-x86_64-cpython-311/tiktoken_ext
copying tiktoken_ext/openai_public.py -> build/lib.linux-x86_64-cpython-311/tiktoken_ext
running egg_info
writing tiktoken.egg-info/PKG-INFO
writing dependency_links to tiktoken.egg-info/dependency_links.txt
writing requirements to tiktoken.egg-info/requires.txt
writing top-level names to tiktoken.egg-info/top_level.txt
reading manifest file 'tiktoken.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'Makefile'
adding license file 'LICENSE'
writing manifest file 'tiktoken.egg-info/SOURCES.txt'
copying tiktoken/py.typed -> build/lib.linux-x86_64-cpython-311/tiktoken
running build_ext
running build_rust
error: can't find Rust compiler
If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
To update pip, run:
pip install --upgrade pip
and then retry package installation.
If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for tiktoken
Failed to build tiktoken
ERROR: Could not build wheels for tiktoken, which is required to install pyproject.toml-based projects
[notice] A new release of pip is available: 24.0 -> 25.2
[notice] To update, run: pip install --upgrade pip
bash-4.2#
- Observe that pip tries to build from source instead of downloading a prebuilt wheel.
Expected Behavior
A precompiled wheel (manylinux_2014_x86_64 or manylinux_2_17_x86_64) should be available for tiktoken 0.12.0, so it installs without requiring a Rust compiler in standard Linux build environments (Amazon Linux 2, AWS Lambda, AWS SAM build).
Actual Behavior
The tiktoken 0.12.0 version only ships wheels for newer manylinux_2_28 (Amazon Linux 2023) and musllinux.
In Amazon Linux 2-based CI images, pip downloads the source tarball and fails with:
...
error: can't find Rust compiler
...
Environment
OS: Amazon Linux 2 (AWS SAM build image public.ecr.aws/sam/build-python3.11:latest)
Python: 3.11
tiktoken: 0.12.0
pip: 24.0
Rust: not installed
bash-4.2# bash -lc 'echo "OS:"; cat /etc/os-release; echo; ldd --version | head -1'
OS:
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"
SUPPORT_END="2026-06-30"
VARIANT_ID="202507310614-2.0.1488.0"
ldd (GNU libc) 2.26
bash-4.2# python -m pip debug --verbose | sed -n "/Compatible tags/,/Package index/p"
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
Compatible tags: 664
cp311-cp311-manylinux_2_26_x86_64
cp311-cp311-manylinux_2_25_x86_64
cp311-cp311-manylinux_2_24_x86_64
cp311-cp311-manylinux_2_23_x86_64
cp311-cp311-manylinux_2_22_x86_64
cp311-cp311-manylinux_2_21_x86_64
cp311-cp311-manylinux_2_20_x86_64
cp311-cp311-manylinux_2_19_x86_64
cp311-cp311-manylinux_2_18_x86_64
cp311-cp311-manylinux_2_17_x86_64
cp311-cp311-manylinux2014_x86_64
cp311-cp311-manylinux_2_16_x86_64
cp311-cp311-manylinux_2_15_x86_64
cp311-cp311-manylinux_2_14_x86_64
cp311-cp311-manylinux_2_13_x86_64
cp311-cp311-manylinux_2_12_x86_64
cp311-cp311-manylinux2010_x86_64
cp311-cp311-manylinux_2_11_x86_64
cp311-cp311-manylinux_2_10_x86_64
cp311-cp311-manylinux_2_9_x86_64
cp311-cp311-manylinux_2_8_x86_64
cp311-cp311-manylinux_2_7_x86_64
cp311-cp311-manylinux_2_6_x86_64
cp311-cp311-manylinux_2_5_x86_64
cp311-cp311-manylinux1_x86_64
cp311-cp311-linux_x86_64
cp311-abi3-manylinux_2_26_x86_64
cp311-abi3-manylinux_2_25_x86_64
cp311-abi3-manylinux_2_24_x86_64
cp311-abi3-manylinux_2_23_x86_64
cp311-abi3-manylinux_2_22_x86_64
cp311-abi3-manylinux_2_21_x86_64
cp311-abi3-manylinux_2_20_x86_64
cp311-abi3-manylinux_2_19_x86_64
cp311-abi3-manylinux_2_18_x86_64
cp311-abi3-manylinux_2_17_x86_64
cp311-abi3-manylinux2014_x86_64
cp311-abi3-manylinux_2_16_x86_64
cp311-abi3-manylinux_2_15_x86_64
cp311-abi3-manylinux_2_14_x86_64
cp311-abi3-manylinux_2_13_x86_64
cp311-abi3-manylinux_2_12_x86_64
cp311-abi3-manylinux2010_x86_64
cp311-abi3-manylinux_2_11_x86_64
cp311-abi3-manylinux_2_10_x86_64
cp311-abi3-manylinux_2_9_x86_64
cp311-abi3-manylinux_2_8_x86_64
cp311-abi3-manylinux_2_7_x86_64
cp311-abi3-manylinux_2_6_x86_64
cp311-abi3-manylinux_2_5_x86_64
cp311-abi3-manylinux1_x86_64
cp311-abi3-linux_x86_64
cp311-none-manylinux_2_26_x86_64
cp311-none-manylinux_2_25_x86_64
cp311-none-manylinux_2_24_x86_64
cp311-none-manylinux_2_23_x86_64
cp311-none-manylinux_2_22_x86_64
cp311-none-manylinux_2_21_x86_64
cp311-none-manylinux_2_20_x86_64
cp311-none-manylinux_2_19_x86_64
cp311-none-manylinux_2_18_x86_64
cp311-none-manylinux_2_17_x86_64
cp311-none-manylinux2014_x86_64
cp311-none-manylinux_2_16_x86_64
cp311-none-manylinux_2_15_x86_64
cp311-none-manylinux_2_14_x86_64
cp311-none-manylinux_2_13_x86_64
cp311-none-manylinux_2_12_x86_64
cp311-none-manylinux2010_x86_64
cp311-none-manylinux_2_11_x86_64
cp311-none-manylinux_2_10_x86_64
cp311-none-manylinux_2_9_x86_64
cp311-none-manylinux_2_8_x86_64
cp311-none-manylinux_2_7_x86_64
cp311-none-manylinux_2_6_x86_64
cp311-none-manylinux_2_5_x86_64
cp311-none-manylinux1_x86_64
cp311-none-linux_x86_64
cp310-abi3-manylinux_2_26_x86_64
cp310-abi3-manylinux_2_25_x86_64
cp310-abi3-manylinux_2_24_x86_64
cp310-abi3-manylinux_2_23_x86_64
cp310-abi3-manylinux_2_22_x86_64
cp310-abi3-manylinux_2_21_x86_64
cp310-abi3-manylinux_2_20_x86_64
cp310-abi3-manylinux_2_19_x86_64
cp310-abi3-manylinux_2_18_x86_64
cp310-abi3-manylinux_2_17_x86_64
cp310-abi3-manylinux2014_x86_64
cp310-abi3-manylinux_2_16_x86_64
cp310-abi3-manylinux_2_15_x86_64
cp310-abi3-manylinux_2_14_x86_64
cp310-abi3-manylinux_2_13_x86_64
cp310-abi3-manylinux_2_12_x86_64
cp310-abi3-manylinux2010_x86_64
cp310-abi3-manylinux_2_11_x86_64
cp310-abi3-manylinux_2_10_x86_64
cp310-abi3-manylinux_2_9_x86_64
cp310-abi3-manylinux_2_8_x86_64
cp310-abi3-manylinux_2_7_x86_64
cp310-abi3-manylinux_2_6_x86_64
cp310-abi3-manylinux_2_5_x86_64
cp310-abi3-manylinux1_x86_64
cp310-abi3-linux_x86_64
cp39-abi3-manylinux_2_26_x86_64
cp39-abi3-manylinux_2_25_x86_64
cp39-abi3-manylinux_2_24_x86_64
cp39-abi3-manylinux_2_23_x86_64
cp39-abi3-manylinux_2_22_x86_64
cp39-abi3-manylinux_2_21_x86_64
cp39-abi3-manylinux_2_20_x86_64
cp39-abi3-manylinux_2_19_x86_64
cp39-abi3-manylinux_2_18_x86_64
cp39-abi3-manylinux_2_17_x86_64
cp39-abi3-manylinux2014_x86_64
cp39-abi3-manylinux_2_16_x86_64
cp39-abi3-manylinux_2_15_x86_64
cp39-abi3-manylinux_2_14_x86_64
cp39-abi3-manylinux_2_13_x86_64
cp39-abi3-manylinux_2_12_x86_64
cp39-abi3-manylinux2010_x86_64
cp39-abi3-manylinux_2_11_x86_64
cp39-abi3-manylinux_2_10_x86_64
cp39-abi3-manylinux_2_9_x86_64
cp39-abi3-manylinux_2_8_x86_64
cp39-abi3-manylinux_2_7_x86_64
cp39-abi3-manylinux_2_6_x86_64
cp39-abi3-manylinux_2_5_x86_64
cp39-abi3-manylinux1_x86_64
cp39-abi3-linux_x86_64
cp38-abi3-manylinux_2_26_x86_64
cp38-abi3-manylinux_2_25_x86_64
cp38-abi3-manylinux_2_24_x86_64
cp38-abi3-manylinux_2_23_x86_64
cp38-abi3-manylinux_2_22_x86_64
cp38-abi3-manylinux_2_21_x86_64
cp38-abi3-manylinux_2_20_x86_64
cp38-abi3-manylinux_2_19_x86_64
cp38-abi3-manylinux_2_18_x86_64
cp38-abi3-manylinux_2_17_x86_64
cp38-abi3-manylinux2014_x86_64
cp38-abi3-manylinux_2_16_x86_64
cp38-abi3-manylinux_2_15_x86_64
cp38-abi3-manylinux_2_14_x86_64
cp38-abi3-manylinux_2_13_x86_64
cp38-abi3-manylinux_2_12_x86_64
cp38-abi3-manylinux2010_x86_64
cp38-abi3-manylinux_2_11_x86_64
cp38-abi3-manylinux_2_10_x86_64
cp38-abi3-manylinux_2_9_x86_64
cp38-abi3-manylinux_2_8_x86_64
cp38-abi3-manylinux_2_7_x86_64
cp38-abi3-manylinux_2_6_x86_64
cp38-abi3-manylinux_2_5_x86_64
cp38-abi3-manylinux1_x86_64
cp38-abi3-linux_x86_64
cp37-abi3-manylinux_2_26_x86_64
cp37-abi3-manylinux_2_25_x86_64
cp37-abi3-manylinux_2_24_x86_64
cp37-abi3-manylinux_2_23_x86_64
cp37-abi3-manylinux_2_22_x86_64
cp37-abi3-manylinux_2_21_x86_64
cp37-abi3-manylinux_2_20_x86_64
cp37-abi3-manylinux_2_19_x86_64
cp37-abi3-manylinux_2_18_x86_64
cp37-abi3-manylinux_2_17_x86_64
cp37-abi3-manylinux2014_x86_64
cp37-abi3-manylinux_2_16_x86_64
cp37-abi3-manylinux_2_15_x86_64
cp37-abi3-manylinux_2_14_x86_64
cp37-abi3-manylinux_2_13_x86_64
cp37-abi3-manylinux_2_12_x86_64
cp37-abi3-manylinux2010_x86_64
cp37-abi3-manylinux_2_11_x86_64
cp37-abi3-manylinux_2_10_x86_64
cp37-abi3-manylinux_2_9_x86_64
cp37-abi3-manylinux_2_8_x86_64
cp37-abi3-manylinux_2_7_x86_64
cp37-abi3-manylinux_2_6_x86_64
cp37-abi3-manylinux_2_5_x86_64
cp37-abi3-manylinux1_x86_64
cp37-abi3-linux_x86_64
cp36-abi3-manylinux_2_26_x86_64
cp36-abi3-manylinux_2_25_x86_64
cp36-abi3-manylinux_2_24_x86_64
cp36-abi3-manylinux_2_23_x86_64
cp36-abi3-manylinux_2_22_x86_64
cp36-abi3-manylinux_2_21_x86_64
cp36-abi3-manylinux_2_20_x86_64
cp36-abi3-manylinux_2_19_x86_64
cp36-abi3-manylinux_2_18_x86_64
cp36-abi3-manylinux_2_17_x86_64
cp36-abi3-manylinux2014_x86_64
cp36-abi3-manylinux_2_16_x86_64
cp36-abi3-manylinux_2_15_x86_64
cp36-abi3-manylinux_2_14_x86_64
cp36-abi3-manylinux_2_13_x86_64
cp36-abi3-manylinux_2_12_x86_64
cp36-abi3-manylinux2010_x86_64
cp36-abi3-manylinux_2_11_x86_64
cp36-abi3-manylinux_2_10_x86_64
cp36-abi3-manylinux_2_9_x86_64
cp36-abi3-manylinux_2_8_x86_64
cp36-abi3-manylinux_2_7_x86_64
cp36-abi3-manylinux_2_6_x86_64
cp36-abi3-manylinux_2_5_x86_64
cp36-abi3-manylinux1_x86_64
cp36-abi3-linux_x86_64
cp35-abi3-manylinux_2_26_x86_64
cp35-abi3-manylinux_2_25_x86_64
cp35-abi3-manylinux_2_24_x86_64
cp35-abi3-manylinux_2_23_x86_64
cp35-abi3-manylinux_2_22_x86_64
cp35-abi3-manylinux_2_21_x86_64
cp35-abi3-manylinux_2_20_x86_64
cp35-abi3-manylinux_2_19_x86_64
cp35-abi3-manylinux_2_18_x86_64
cp35-abi3-manylinux_2_17_x86_64
cp35-abi3-manylinux2014_x86_64
cp35-abi3-manylinux_2_16_x86_64
cp35-abi3-manylinux_2_15_x86_64
cp35-abi3-manylinux_2_14_x86_64
cp35-abi3-manylinux_2_13_x86_64
cp35-abi3-manylinux_2_12_x86_64
cp35-abi3-manylinux2010_x86_64
cp35-abi3-manylinux_2_11_x86_64
cp35-abi3-manylinux_2_10_x86_64
cp35-abi3-manylinux_2_9_x86_64
cp35-abi3-manylinux_2_8_x86_64
cp35-abi3-manylinux_2_7_x86_64
cp35-abi3-manylinux_2_6_x86_64
cp35-abi3-manylinux_2_5_x86_64
cp35-abi3-manylinux1_x86_64
cp35-abi3-linux_x86_64
cp34-abi3-manylinux_2_26_x86_64
cp34-abi3-manylinux_2_25_x86_64
cp34-abi3-manylinux_2_24_x86_64
cp34-abi3-manylinux_2_23_x86_64
cp34-abi3-manylinux_2_22_x86_64
cp34-abi3-manylinux_2_21_x86_64
cp34-abi3-manylinux_2_20_x86_64
cp34-abi3-manylinux_2_19_x86_64
cp34-abi3-manylinux_2_18_x86_64
cp34-abi3-manylinux_2_17_x86_64
cp34-abi3-manylinux2014_x86_64
cp34-abi3-manylinux_2_16_x86_64
cp34-abi3-manylinux_2_15_x86_64
cp34-abi3-manylinux_2_14_x86_64
cp34-abi3-manylinux_2_13_x86_64
cp34-abi3-manylinux_2_12_x86_64
cp34-abi3-manylinux2010_x86_64
cp34-abi3-manylinux_2_11_x86_64
cp34-abi3-manylinux_2_10_x86_64
cp34-abi3-manylinux_2_9_x86_64
cp34-abi3-manylinux_2_8_x86_64
cp34-abi3-manylinux_2_7_x86_64
cp34-abi3-manylinux_2_6_x86_64
cp34-abi3-manylinux_2_5_x86_64
cp34-abi3-manylinux1_x86_64
cp34-abi3-linux_x86_64
cp33-abi3-manylinux_2_26_x86_64
cp33-abi3-manylinux_2_25_x86_64
cp33-abi3-manylinux_2_24_x86_64
cp33-abi3-manylinux_2_23_x86_64
cp33-abi3-manylinux_2_22_x86_64
cp33-abi3-manylinux_2_21_x86_64
cp33-abi3-manylinux_2_20_x86_64
cp33-abi3-manylinux_2_19_x86_64
cp33-abi3-manylinux_2_18_x86_64
cp33-abi3-manylinux_2_17_x86_64
cp33-abi3-manylinux2014_x86_64
cp33-abi3-manylinux_2_16_x86_64
cp33-abi3-manylinux_2_15_x86_64
cp33-abi3-manylinux_2_14_x86_64
cp33-abi3-manylinux_2_13_x86_64
cp33-abi3-manylinux_2_12_x86_64
cp33-abi3-manylinux2010_x86_64
cp33-abi3-manylinux_2_11_x86_64
cp33-abi3-manylinux_2_10_x86_64
cp33-abi3-manylinux_2_9_x86_64
cp33-abi3-manylinux_2_8_x86_64
cp33-abi3-manylinux_2_7_x86_64
cp33-abi3-manylinux_2_6_x86_64
cp33-abi3-manylinux_2_5_x86_64
cp33-abi3-manylinux1_x86_64
cp33-abi3-linux_x86_64
cp32-abi3-manylinux_2_26_x86_64
cp32-abi3-manylinux_2_25_x86_64
cp32-abi3-manylinux_2_24_x86_64
cp32-abi3-manylinux_2_23_x86_64
cp32-abi3-manylinux_2_22_x86_64
cp32-abi3-manylinux_2_21_x86_64
cp32-abi3-manylinux_2_20_x86_64
cp32-abi3-manylinux_2_19_x86_64
cp32-abi3-manylinux_2_18_x86_64
cp32-abi3-manylinux_2_17_x86_64
cp32-abi3-manylinux2014_x86_64
cp32-abi3-manylinux_2_16_x86_64
cp32-abi3-manylinux_2_15_x86_64
cp32-abi3-manylinux_2_14_x86_64
cp32-abi3-manylinux_2_13_x86_64
cp32-abi3-manylinux_2_12_x86_64
cp32-abi3-manylinux2010_x86_64
cp32-abi3-manylinux_2_11_x86_64
cp32-abi3-manylinux_2_10_x86_64
cp32-abi3-manylinux_2_9_x86_64
cp32-abi3-manylinux_2_8_x86_64
cp32-abi3-manylinux_2_7_x86_64
cp32-abi3-manylinux_2_6_x86_64
cp32-abi3-manylinux_2_5_x86_64
cp32-abi3-manylinux1_x86_64
cp32-abi3-linux_x86_64
py311-none-manylinux_2_26_x86_64
py311-none-manylinux_2_25_x86_64
py311-none-manylinux_2_24_x86_64
py311-none-manylinux_2_23_x86_64
py311-none-manylinux_2_22_x86_64
py311-none-manylinux_2_21_x86_64
py311-none-manylinux_2_20_x86_64
py311-none-manylinux_2_19_x86_64
py311-none-manylinux_2_18_x86_64
py311-none-manylinux_2_17_x86_64
py311-none-manylinux2014_x86_64
py311-none-manylinux_2_16_x86_64
py311-none-manylinux_2_15_x86_64
py311-none-manylinux_2_14_x86_64
py311-none-manylinux_2_13_x86_64
py311-none-manylinux_2_12_x86_64
py311-none-manylinux2010_x86_64
py311-none-manylinux_2_11_x86_64
py311-none-manylinux_2_10_x86_64
py311-none-manylinux_2_9_x86_64
py311-none-manylinux_2_8_x86_64
py311-none-manylinux_2_7_x86_64
py311-none-manylinux_2_6_x86_64
py311-none-manylinux_2_5_x86_64
py311-none-manylinux1_x86_64
py311-none-linux_x86_64
py3-none-manylinux_2_26_x86_64
py3-none-manylinux_2_25_x86_64
py3-none-manylinux_2_24_x86_64
py3-none-manylinux_2_23_x86_64
py3-none-manylinux_2_22_x86_64
py3-none-manylinux_2_21_x86_64
py3-none-manylinux_2_20_x86_64
py3-none-manylinux_2_19_x86_64
py3-none-manylinux_2_18_x86_64
py3-none-manylinux_2_17_x86_64
py3-none-manylinux2014_x86_64
py3-none-manylinux_2_16_x86_64
py3-none-manylinux_2_15_x86_64
py3-none-manylinux_2_14_x86_64
py3-none-manylinux_2_13_x86_64
py3-none-manylinux_2_12_x86_64
py3-none-manylinux2010_x86_64
py3-none-manylinux_2_11_x86_64
py3-none-manylinux_2_10_x86_64
py3-none-manylinux_2_9_x86_64
py3-none-manylinux_2_8_x86_64
py3-none-manylinux_2_7_x86_64
py3-none-manylinux_2_6_x86_64
py3-none-manylinux_2_5_x86_64
py3-none-manylinux1_x86_64
py3-none-linux_x86_64
py310-none-manylinux_2_26_x86_64
py310-none-manylinux_2_25_x86_64
py310-none-manylinux_2_24_x86_64
py310-none-manylinux_2_23_x86_64
py310-none-manylinux_2_22_x86_64
py310-none-manylinux_2_21_x86_64
py310-none-manylinux_2_20_x86_64
py310-none-manylinux_2_19_x86_64
py310-none-manylinux_2_18_x86_64
py310-none-manylinux_2_17_x86_64
py310-none-manylinux2014_x86_64
py310-none-manylinux_2_16_x86_64
py310-none-manylinux_2_15_x86_64
py310-none-manylinux_2_14_x86_64
py310-none-manylinux_2_13_x86_64
py310-none-manylinux_2_12_x86_64
py310-none-manylinux2010_x86_64
py310-none-manylinux_2_11_x86_64
py310-none-manylinux_2_10_x86_64
py310-none-manylinux_2_9_x86_64
py310-none-manylinux_2_8_x86_64
py310-none-manylinux_2_7_x86_64
py310-none-manylinux_2_6_x86_64
py310-none-manylinux_2_5_x86_64
py310-none-manylinux1_x86_64
py310-none-linux_x86_64
py39-none-manylinux_2_26_x86_64
py39-none-manylinux_2_25_x86_64
py39-none-manylinux_2_24_x86_64
py39-none-manylinux_2_23_x86_64
py39-none-manylinux_2_22_x86_64
py39-none-manylinux_2_21_x86_64
py39-none-manylinux_2_20_x86_64
py39-none-manylinux_2_19_x86_64
py39-none-manylinux_2_18_x86_64
py39-none-manylinux_2_17_x86_64
py39-none-manylinux2014_x86_64
py39-none-manylinux_2_16_x86_64
py39-none-manylinux_2_15_x86_64
py39-none-manylinux_2_14_x86_64
py39-none-manylinux_2_13_x86_64
py39-none-manylinux_2_12_x86_64
py39-none-manylinux2010_x86_64
py39-none-manylinux_2_11_x86_64
py39-none-manylinux_2_10_x86_64
py39-none-manylinux_2_9_x86_64
py39-none-manylinux_2_8_x86_64
py39-none-manylinux_2_7_x86_64
py39-none-manylinux_2_6_x86_64
py39-none-manylinux_2_5_x86_64
py39-none-manylinux1_x86_64
py39-none-linux_x86_64
py38-none-manylinux_2_26_x86_64
py38-none-manylinux_2_25_x86_64
py38-none-manylinux_2_24_x86_64
py38-none-manylinux_2_23_x86_64
py38-none-manylinux_2_22_x86_64
py38-none-manylinux_2_21_x86_64
py38-none-manylinux_2_20_x86_64
py38-none-manylinux_2_19_x86_64
py38-none-manylinux_2_18_x86_64
py38-none-manylinux_2_17_x86_64
py38-none-manylinux2014_x86_64
py38-none-manylinux_2_16_x86_64
py38-none-manylinux_2_15_x86_64
py38-none-manylinux_2_14_x86_64
py38-none-manylinux_2_13_x86_64
py38-none-manylinux_2_12_x86_64
py38-none-manylinux2010_x86_64
py38-none-manylinux_2_11_x86_64
py38-none-manylinux_2_10_x86_64
py38-none-manylinux_2_9_x86_64
py38-none-manylinux_2_8_x86_64
py38-none-manylinux_2_7_x86_64
py38-none-manylinux_2_6_x86_64
py38-none-manylinux_2_5_x86_64
py38-none-manylinux1_x86_64
py38-none-linux_x86_64
py37-none-manylinux_2_26_x86_64
py37-none-manylinux_2_25_x86_64
py37-none-manylinux_2_24_x86_64
py37-none-manylinux_2_23_x86_64
py37-none-manylinux_2_22_x86_64
py37-none-manylinux_2_21_x86_64
py37-none-manylinux_2_20_x86_64
py37-none-manylinux_2_19_x86_64
py37-none-manylinux_2_18_x86_64
py37-none-manylinux_2_17_x86_64
py37-none-manylinux2014_x86_64
py37-none-manylinux_2_16_x86_64
py37-none-manylinux_2_15_x86_64
py37-none-manylinux_2_14_x86_64
py37-none-manylinux_2_13_x86_64
py37-none-manylinux_2_12_x86_64
py37-none-manylinux2010_x86_64
py37-none-manylinux_2_11_x86_64
py37-none-manylinux_2_10_x86_64
py37-none-manylinux_2_9_x86_64
py37-none-manylinux_2_8_x86_64
py37-none-manylinux_2_7_x86_64
py37-none-manylinux_2_6_x86_64
py37-none-manylinux_2_5_x86_64
py37-none-manylinux1_x86_64
py37-none-linux_x86_64
py36-none-manylinux_2_26_x86_64
py36-none-manylinux_2_25_x86_64
py36-none-manylinux_2_24_x86_64
py36-none-manylinux_2_23_x86_64
py36-none-manylinux_2_22_x86_64
py36-none-manylinux_2_21_x86_64
py36-none-manylinux_2_20_x86_64
py36-none-manylinux_2_19_x86_64
py36-none-manylinux_2_18_x86_64
py36-none-manylinux_2_17_x86_64
py36-none-manylinux2014_x86_64
py36-none-manylinux_2_16_x86_64
py36-none-manylinux_2_15_x86_64
py36-none-manylinux_2_14_x86_64
py36-none-manylinux_2_13_x86_64
py36-none-manylinux_2_12_x86_64
py36-none-manylinux2010_x86_64
py36-none-manylinux_2_11_x86_64
py36-none-manylinux_2_10_x86_64
py36-none-manylinux_2_9_x86_64
py36-none-manylinux_2_8_x86_64
py36-none-manylinux_2_7_x86_64
py36-none-manylinux_2_6_x86_64
py36-none-manylinux_2_5_x86_64
py36-none-manylinux1_x86_64
py36-none-linux_x86_64
py35-none-manylinux_2_26_x86_64
py35-none-manylinux_2_25_x86_64
py35-none-manylinux_2_24_x86_64
py35-none-manylinux_2_23_x86_64
py35-none-manylinux_2_22_x86_64
py35-none-manylinux_2_21_x86_64
py35-none-manylinux_2_20_x86_64
py35-none-manylinux_2_19_x86_64
py35-none-manylinux_2_18_x86_64
py35-none-manylinux_2_17_x86_64
py35-none-manylinux2014_x86_64
py35-none-manylinux_2_16_x86_64
py35-none-manylinux_2_15_x86_64
py35-none-manylinux_2_14_x86_64
py35-none-manylinux_2_13_x86_64
py35-none-manylinux_2_12_x86_64
py35-none-manylinux2010_x86_64
py35-none-manylinux_2_11_x86_64
py35-none-manylinux_2_10_x86_64
py35-none-manylinux_2_9_x86_64
py35-none-manylinux_2_8_x86_64
py35-none-manylinux_2_7_x86_64
py35-none-manylinux_2_6_x86_64
py35-none-manylinux_2_5_x86_64
py35-none-manylinux1_x86_64
py35-none-linux_x86_64
py34-none-manylinux_2_26_x86_64
py34-none-manylinux_2_25_x86_64
py34-none-manylinux_2_24_x86_64
py34-none-manylinux_2_23_x86_64
py34-none-manylinux_2_22_x86_64
py34-none-manylinux_2_21_x86_64
py34-none-manylinux_2_20_x86_64
py34-none-manylinux_2_19_x86_64
py34-none-manylinux_2_18_x86_64
py34-none-manylinux_2_17_x86_64
py34-none-manylinux2014_x86_64
py34-none-manylinux_2_16_x86_64
py34-none-manylinux_2_15_x86_64
py34-none-manylinux_2_14_x86_64
py34-none-manylinux_2_13_x86_64
py34-none-manylinux_2_12_x86_64
py34-none-manylinux2010_x86_64
py34-none-manylinux_2_11_x86_64
py34-none-manylinux_2_10_x86_64
py34-none-manylinux_2_9_x86_64
py34-none-manylinux_2_8_x86_64
py34-none-manylinux_2_7_x86_64
py34-none-manylinux_2_6_x86_64
py34-none-manylinux_2_5_x86_64
py34-none-manylinux1_x86_64
py34-none-linux_x86_64
py33-none-manylinux_2_26_x86_64
py33-none-manylinux_2_25_x86_64
py33-none-manylinux_2_24_x86_64
py33-none-manylinux_2_23_x86_64
py33-none-manylinux_2_22_x86_64
py33-none-manylinux_2_21_x86_64
py33-none-manylinux_2_20_x86_64
py33-none-manylinux_2_19_x86_64
py33-none-manylinux_2_18_x86_64
py33-none-manylinux_2_17_x86_64
py33-none-manylinux2014_x86_64
py33-none-manylinux_2_16_x86_64
py33-none-manylinux_2_15_x86_64
py33-none-manylinux_2_14_x86_64
py33-none-manylinux_2_13_x86_64
py33-none-manylinux_2_12_x86_64
py33-none-manylinux2010_x86_64
py33-none-manylinux_2_11_x86_64
py33-none-manylinux_2_10_x86_64
py33-none-manylinux_2_9_x86_64
py33-none-manylinux_2_8_x86_64
py33-none-manylinux_2_7_x86_64
py33-none-manylinux_2_6_x86_64
py33-none-manylinux_2_5_x86_64
py33-none-manylinux1_x86_64
py33-none-linux_x86_64
py32-none-manylinux_2_26_x86_64
py32-none-manylinux_2_25_x86_64
py32-none-manylinux_2_24_x86_64
py32-none-manylinux_2_23_x86_64
py32-none-manylinux_2_22_x86_64
py32-none-manylinux_2_21_x86_64
py32-none-manylinux_2_20_x86_64
py32-none-manylinux_2_19_x86_64
py32-none-manylinux_2_18_x86_64
py32-none-manylinux_2_17_x86_64
py32-none-manylinux2014_x86_64
py32-none-manylinux_2_16_x86_64
py32-none-manylinux_2_15_x86_64
py32-none-manylinux_2_14_x86_64
py32-none-manylinux_2_13_x86_64
py32-none-manylinux_2_12_x86_64
py32-none-manylinux2010_x86_64
py32-none-manylinux_2_11_x86_64
py32-none-manylinux_2_10_x86_64
py32-none-manylinux_2_9_x86_64
py32-none-manylinux_2_8_x86_64
py32-none-manylinux_2_7_x86_64
py32-none-manylinux_2_6_x86_64
py32-none-manylinux_2_5_x86_64
py32-none-manylinux1_x86_64
py32-none-linux_x86_64
py31-none-manylinux_2_26_x86_64
py31-none-manylinux_2_25_x86_64
py31-none-manylinux_2_24_x86_64
py31-none-manylinux_2_23_x86_64
py31-none-manylinux_2_22_x86_64
py31-none-manylinux_2_21_x86_64
py31-none-manylinux_2_20_x86_64
py31-none-manylinux_2_19_x86_64
py31-none-manylinux_2_18_x86_64
py31-none-manylinux_2_17_x86_64
py31-none-manylinux2014_x86_64
py31-none-manylinux_2_16_x86_64
py31-none-manylinux_2_15_x86_64
py31-none-manylinux_2_14_x86_64
py31-none-manylinux_2_13_x86_64
py31-none-manylinux_2_12_x86_64
py31-none-manylinux2010_x86_64
py31-none-manylinux_2_11_x86_64
py31-none-manylinux_2_10_x86_64
py31-none-manylinux_2_9_x86_64
py31-none-manylinux_2_8_x86_64
py31-none-manylinux_2_7_x86_64
py31-none-manylinux_2_6_x86_64
py31-none-manylinux_2_5_x86_64
py31-none-manylinux1_x86_64
py31-none-linux_x86_64
py30-none-manylinux_2_26_x86_64
py30-none-manylinux_2_25_x86_64
py30-none-manylinux_2_24_x86_64
py30-none-manylinux_2_23_x86_64
py30-none-manylinux_2_22_x86_64
py30-none-manylinux_2_21_x86_64
py30-none-manylinux_2_20_x86_64
py30-none-manylinux_2_19_x86_64
py30-none-manylinux_2_18_x86_64
py30-none-manylinux_2_17_x86_64
py30-none-manylinux2014_x86_64
py30-none-manylinux_2_16_x86_64
py30-none-manylinux_2_15_x86_64
py30-none-manylinux_2_14_x86_64
py30-none-manylinux_2_13_x86_64
py30-none-manylinux_2_12_x86_64
py30-none-manylinux2010_x86_64
py30-none-manylinux_2_11_x86_64
py30-none-manylinux_2_10_x86_64
py30-none-manylinux_2_9_x86_64
py30-none-manylinux_2_8_x86_64
py30-none-manylinux_2_7_x86_64
py30-none-manylinux_2_6_x86_64
py30-none-manylinux_2_5_x86_64
py30-none-manylinux1_x86_64
py30-none-linux_x86_64
cp311-none-any
py311-none-any
py3-none-any
py310-none-any
py39-none-any
py38-none-any
py37-none-any
py36-none-any
py35-none-any
py34-none-any
py33-none-any
py32-none-any
py31-none-any
py30-none-any
bash-4.2#
Description
Starting from tiktoken==0.12.0, the release no longer provides precompiled manylinux2014_x86_64 wheels, only newer manylinux_2_28 (Amazon Linux 2023) and musllinux variants.
This change breaks CI/CD pipelines and AWS Lambda build environments still based on Amazon Linux 2 (glibc 2.26/2.27, manylinux_2_17), where pip must compile from source — which fails because no Rust compiler is available.
This issue affects not only direct users of tiktoken, but also indirect dependents such as:
Both of these packages declare a dependency range like:
As a result, any build that installs these libraries will automatically pull the latest tiktoken (currently 0.12.0) — causing failures in CI environments or Docker images based on Amazon Linux 2, such as the official AWS SAM build images.
This makes otherwise functional codebases fail during dependency resolution, even though tiktoken==0.11.0 worked correctly.
Steps to Reproduce:
public.ecr.aws/sam/build-python3.11:latest(Amazon Linux 2 / manylinux_2_17 compatible)
Expected Behavior
A precompiled wheel (manylinux_2014_x86_64 or manylinux_2_17_x86_64) should be available for tiktoken 0.12.0, so it installs without requiring a Rust compiler in standard Linux build environments (Amazon Linux 2, AWS Lambda, AWS SAM build).
Actual Behavior
The tiktoken 0.12.0 version only ships wheels for newer manylinux_2_28 (Amazon Linux 2023) and musllinux.
In Amazon Linux 2-based CI images, pip downloads the source tarball and fails with:
Environment
OS: Amazon Linux 2 (AWS SAM build image public.ecr.aws/sam/build-python3.11:latest)
Python: 3.11
tiktoken: 0.12.0
pip: 24.0
Rust: not installed