Skip to content

Commit a8b8eb4

Browse files
matthewfeickertrasbt
matthewfeickert
and
rasbt
authored
feat: Add pixi environment (#534)
* feat: Add pixi environment * Add pixi manifest pixi.toml for Linux x86, macOS arm64, Windows 64. * ci: Update CI workflow and unify to one * Enable workflow dispatch. * Add concurrency limits. * Use pixi for CI workflow. * Unify to a single workflow for all OS tested * feat: Add pixi lock file * Ensure tensorflow-cpu installed on Windows * fix package check * fix package check * simplification plus uv and pip runners * some fixes to pixi and pip * create pixi.lock * fix pixi.lock issue * another attempt trying to fix get_packages * another attempt trying to fix get_packages * clean up python_environment_check.py * updated runner and docs * use bash * proper env activiation * proper env activiation --------- Co-authored-by: rasbt <[email protected]>
1 parent 16738b6 commit a8b8eb4

14 files changed

+13165
-218
lines changed

‎.github/workflows/basic-tests-linux.yml

-55
This file was deleted.

‎.github/workflows/basic-tests-macos.yml

-55
This file was deleted.

‎.github/workflows/basic-tests-pip.yml

+61
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
name: Code tests (plain pip)
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
paths:
7+
- '**/*.py'
8+
- '**/*.ipynb'
9+
- '**/*.yaml'
10+
- '**/*.yml'
11+
- '**/*.sh'
12+
pull_request:
13+
branches: [ main ]
14+
paths:
15+
- '**/*.py'
16+
- '**/*.ipynb'
17+
- '**/*.yaml'
18+
- '**/*.yml'
19+
- '**/*.sh'
20+
workflow_dispatch:
21+
22+
concurrency:
23+
group: ${{ github.workflow }}-${{ github.ref }}
24+
cancel-in-progress: true
25+
26+
jobs:
27+
pip-tests:
28+
name: Pip Tests (Ubuntu Only)
29+
runs-on: ubuntu-latest
30+
steps:
31+
- uses: actions/checkout@v4
32+
33+
- name: Set up Python
34+
uses: actions/setup-python@v5
35+
with:
36+
python-version: "3.10"
37+
38+
- name: Create Virtual Environment and Install Dependencies
39+
run: |
40+
python -m venv .venv
41+
source .venv/bin/activate
42+
pip install --upgrade pip
43+
pip install -r requirements.txt
44+
pip install -r ch05/07_gpt_to_llama/tests/test-requirements-extra.txt
45+
pip install pytest pytest-ruff nbval
46+
47+
- name: Test Selected Python Scripts
48+
run: |
49+
source .venv/bin/activate
50+
pytest --ruff setup/02_installing-python-libraries/tests.py
51+
pytest --ruff ch04/01_main-chapter-code/tests.py
52+
pytest --ruff ch05/01_main-chapter-code/tests.py
53+
pytest --ruff ch05/07_gpt_to_llama/tests/tests.py
54+
pytest --ruff ch06/01_main-chapter-code/tests.py
55+
56+
- name: Validate Selected Jupyter Notebooks
57+
run: |
58+
source .venv/bin/activate
59+
pytest --ruff --nbval ch02/01_main-chapter-code/dataloader.ipynb
60+
pytest --ruff --nbval ch03/01_main-chapter-code/multihead-attention.ipynb
61+
pytest --ruff --nbval ch02/04_bonus_dataloader-intuition/dataloader-intuition.ipynb

‎.github/workflows/basic-tests-pixi.yml

+60
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
name: Code tests (pixi)
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
paths:
7+
- '**/*.py'
8+
- '**/*.ipynb'
9+
- '**/*.yaml'
10+
- '**/*.yml'
11+
- '**/*.sh'
12+
pull_request:
13+
branches: [ main ]
14+
paths:
15+
- '**/*.py'
16+
- '**/*.ipynb'
17+
- '**/*.yaml'
18+
- '**/*.yml'
19+
- '**/*.sh'
20+
workflow_dispatch:
21+
22+
concurrency:
23+
group: ${{ github.workflow }}-${{ github.ref }}
24+
cancel-in-progress: true
25+
26+
jobs:
27+
test:
28+
runs-on: ${{ matrix.os }}
29+
strategy:
30+
matrix:
31+
os: [ubuntu-latest, macos-latest, windows-latest]
32+
33+
steps:
34+
- uses: actions/checkout@v4
35+
36+
- name: Set up pixi (without caching)
37+
uses: prefix-dev/[email protected]
38+
with:
39+
environments: tests
40+
cache: false
41+
42+
- name: List installed packages
43+
run: |
44+
pixi list --environment tests
45+
46+
- name: Test Selected Python Scripts
47+
shell: pixi run --environment tests bash -e {0}
48+
run: |
49+
pytest --ruff setup/02_installing-python-libraries/tests.py
50+
pytest --ruff ch04/01_main-chapter-code/tests.py
51+
pytest --ruff ch05/01_main-chapter-code/tests.py
52+
pytest --ruff ch05/07_gpt_to_llama/tests/tests.py
53+
pytest --ruff ch06/01_main-chapter-code/tests.py
54+
55+
- name: Validate Selected Jupyter Notebooks
56+
shell: pixi run --environment tests bash -e {0}
57+
run: |
58+
pytest --ruff --nbval ch02/01_main-chapter-code/dataloader.ipynb
59+
pytest --ruff --nbval ch03/01_main-chapter-code/multihead-attention.ipynb
60+
pytest --ruff --nbval ch02/04_bonus_dataloader-intuition/dataloader-intuition.ipynb

‎.github/workflows/basic-tests-uv.yml

+66
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
name: Code tests (uv)
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
paths:
7+
- '**/*.py'
8+
- '**/*.ipynb'
9+
- '**/*.yaml'
10+
- '**/*.yml'
11+
- '**/*.sh'
12+
pull_request:
13+
branches: [ main ]
14+
paths:
15+
- '**/*.py'
16+
- '**/*.ipynb'
17+
- '**/*.yaml'
18+
- '**/*.yml'
19+
- '**/*.sh'
20+
workflow_dispatch:
21+
22+
concurrency:
23+
group: ${{ github.workflow }}-${{ github.ref }}
24+
cancel-in-progress: true
25+
26+
jobs:
27+
uv-tests:
28+
name: UV Tests (Matrix OS)
29+
strategy:
30+
matrix:
31+
os: [ ubuntu-latest, macos-latest]
32+
runs-on: ${{ matrix.os }}
33+
steps:
34+
- uses: actions/checkout@v4
35+
36+
- name: Set up Python (uv)
37+
uses: actions/setup-python@v5
38+
with:
39+
python-version: "3.10"
40+
41+
- name: Install uv and dependencies
42+
shell: bash
43+
run: |
44+
curl -LsSf https://astral.sh/uv/install.sh | sh
45+
uv python install 3.10
46+
uv add . --dev
47+
uv pip install -r ch05/07_gpt_to_llama/tests/test-requirements-extra.txt
48+
uv add pytest-ruff nbval
49+
50+
- name: Test Selected Python Scripts (uv)
51+
shell: bash
52+
run: |
53+
source .venv/bin/activate
54+
pytest --ruff setup/02_installing-python-libraries/tests.py
55+
pytest --ruff ch04/01_main-chapter-code/tests.py
56+
pytest --ruff ch05/01_main-chapter-code/tests.py
57+
pytest --ruff ch05/07_gpt_to_llama/tests/tests.py
58+
pytest --ruff ch06/01_main-chapter-code/tests.py
59+
60+
- name: Validate Selected Jupyter Notebooks (uv)
61+
shell: bash
62+
run: |
63+
source .venv/bin/activate
64+
pytest --ruff --nbval ch02/01_main-chapter-code/dataloader.ipynb
65+
pytest --ruff --nbval ch03/01_main-chapter-code/multihead-attention.ipynb
66+
pytest --ruff --nbval ch02/04_bonus_dataloader-intuition/dataloader-intuition.ipynb

‎.github/workflows/basic-tests-windows-pip.yml renamed to ‎.github/workflows/basic-tests-windows-uv-pip.yml

+12-6
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: Code tests (Windows pip)
1+
name: Code tests Windows (uv/pip)
22

33
on:
44
push:
@@ -32,26 +32,32 @@ jobs:
3232
python-version: '3.10'
3333

3434
- name: Install dependencies
35-
shell: pwsh
35+
shell: bash
3636
run: |
37+
export PATH="$HOME/.local/bin:$PATH"
3738
pip install --upgrade pip
38-
pip install -r requirements.txt
39+
pip install uv
40+
uv venv --python=python3.10
41+
source .venv/Scripts/activate
42+
pip install -r requirements.txt # because of dependency issue on Windows when using `uv pip`
3943
pip install tensorflow-io-gcs-filesystem==0.31.0 # Explicit for Windows
4044
pip install -r ch05/07_gpt_to_llama/tests/test-requirements-extra.txt
4145
pip install pytest-ruff nbval
4246
4347
- name: Run Python Tests
44-
shell: pwsh
48+
shell: bash
4549
run: |
50+
source .venv/Scripts/activate
4651
pytest --ruff setup/02_installing-python-libraries/tests.py
4752
pytest --ruff ch04/01_main-chapter-code/tests.py
4853
pytest --ruff ch05/01_main-chapter-code/tests.py
4954
pytest --ruff ch05/07_gpt_to_llama/tests/tests.py
5055
pytest --ruff ch06/01_main-chapter-code/tests.py
5156
5257
- name: Run Jupyter Notebook Tests
53-
shell: pwsh
58+
shell: bash
5459
run: |
60+
source .venv/Scripts/activate
5561
pytest --ruff --nbval ch02/01_main-chapter-code/dataloader.ipynb
5662
pytest --ruff --nbval ch03/01_main-chapter-code/multihead-attention.ipynb
57-
pytest --ruff --nbval ch02/04_bonus_dataloader-intuition/dataloader-intuition.ipynb
63+
pytest --ruff --nbval ch02/04_bonus_dataloader-intuition/dataloader-intuition.ipynb

0 commit comments

Comments
 (0)