Skip to content

Commit 031404b

Browse files
Irfan Ahmadclaude
authored andcommitted
feat: migrate codejail sandbox to its own uv project (uv migration 3/5)
Gives requirements/edx-sandbox/ its own standalone pyproject.toml + uv.lock, independent of the main app's dependency graph (codejail intentionally runs untrusted code in a separate, isolated venv). [tool.edx_lint].uv_constraints holds only the subset of the root constraints relevant to this environment's deps (numpy, lxml, setuptools) -- uv/edx-lint have no cross-project constraint chaining equivalent to pip-compile's "-c ../constraints.txt", so root and sandbox constraints are now independently maintained (documented in requirements/edx-sandbox/README.rst). base.txt is regenerated as a `uv export` compatibility artifact (the README documents it as a supported, if unstable, direct pip-install target). releases/*.txt are untouched -- they're frozen historical snapshots, not part of any active compile loop; README now documents cutting future ones via `uv export` instead of pip-compile. Part of openedx/public-engineering#543 (3 of 5). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent 59e07e8 commit 031404b

6 files changed

Lines changed: 1077 additions & 57 deletions

File tree

Makefile

Lines changed: 23 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ pull_translations: clean_translations ## pull translations via atlas
6464
detect_changed_source_translations: ## check if translation files are up-to-date
6565
i18n_tool changed
6666

67-
pre-requirements: ## install Python requirements for running pip-tools (still needed for requirements/edx-sandbox and scripts/*, which aren't on uv yet)
67+
pre-requirements: ## install Python requirements for running pip-tools (still needed for scripts/*, which aren't on uv yet)
6868
pip install -r requirements/pip-tools.txt
6969

7070
local-requirements: ## no-op; `uv sync` (used by the targets below) already installs -e . itself
@@ -81,19 +81,22 @@ test-requirements: ## install production dependencies plus the testing group (us
8181

8282
requirements: dev-requirements ## install development environment requirements
8383

84-
# requirements/edx-sandbox (codejail's isolated sandbox environment) and the
85-
# scripts/* one-off script directories are not yet migrated to uv (tracked in
86-
# https://github.com/openedx/public-engineering/issues/543) and are still
84+
# The scripts/* one-off script directories are not yet migrated to uv (tracked
85+
# in https://github.com/openedx/public-engineering/issues/543) and are still
8786
# compiled with pip-compile below. Order is important: files must appear
8887
# after everything they include!
8988
REQ_FILES = \
90-
requirements/edx-sandbox/base \
9189
scripts/xblock/requirements \
9290
scripts/user_retirement/requirements/base \
9391
scripts/user_retirement/requirements/testing \
9492
scripts/structures_pruning/requirements/base \
9593
scripts/structures_pruning/requirements/testing
9694

95+
# uv-managed sub-projects, each with their own pyproject.toml + uv.lock,
96+
# independent of the root project's dependency graph.
97+
UV_SUBPROJECTS = \
98+
requirements/edx-sandbox
99+
97100
define COMMON_CONSTRAINTS_TEMP_COMMENT
98101
# This is a temporary solution to override the real common_constraints.txt\n# In edx-lint, until the pyjwt constraint in edx-lint has been removed.\n# See BOM-2721 for more details.\n# Below is the copied and edited version of common_constraints\n
99102
endef
@@ -152,7 +155,21 @@ compile-requirements: pre-requirements ## Regenerate uv.lock for the root projec
152155
export REBUILD=''; \
153156
done
154157

155-
upgrade: $(COMMON_CONSTRAINTS_TXT) ## update all dependencies (uv.lock for the root project, pip-compile for the not-yet-migrated sub-projects) to the latest releases satisfying our constraints
158+
@for d in $(UV_SUBPROJECTS); do \
159+
echo ; \
160+
echo "== $$d ===============================" ; \
161+
uv run --no-project --with edx-lint edx_lint write_uv_constraints $$d/pyproject.toml && \
162+
(cd $$d && uv lock ${UV_LOCK_OPTS}) && \
163+
( \
164+
echo "# GENERATED FILE, DO NOT EDIT DIRECTLY."; \
165+
echo "# Compatibility export for anyone still 'pip install -r $$d/base.txt'"; \
166+
echo "# directly instead of using uv. Source of truth: $$d/pyproject.toml / uv.lock."; \
167+
(cd $$d && uv export --frozen --no-hashes --no-emit-project); \
168+
) > $$d/base.txt \
169+
|| exit 1; \
170+
done
171+
172+
upgrade: $(COMMON_CONSTRAINTS_TXT) ## update all dependencies (uv.lock for the root project and uv sub-projects, pip-compile for the not-yet-migrated sub-projects) to the latest releases satisfying our constraints
156173
$(MAKE) compile-requirements COMPILE_OPTS="--upgrade" UV_LOCK_OPTS="--upgrade"
157174

158175
upgrade-package: ## update just one package to the latest usable release

requirements/edx-sandbox/README.rst

Lines changed: 19 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -10,22 +10,27 @@ within ``<script type="loncapa/python">`` ProblemBlock tags.
1010
Files in this directory
1111
***********************
1212

13-
base.in
14-
=======
13+
pyproject.toml / uv.lock
14+
=========================
1515

16-
This is the current set of requirements or the edx-sandbox environment, and it
17-
is used to generate the ``.txt`` files described below. These requirements
18-
share some constraints with the general edx-platform requirements (via
19-
``../constraints.txt``), but otherwise, they are completely separate.
16+
This is the current set of requirements for the edx-sandbox environment,
17+
managed with `uv <https://docs.astral.sh/uv/>`_ independently of the main
18+
edx-platform application. ``[tool.edx_lint].uv_constraints`` holds only the
19+
subset of the root ``pyproject.toml``'s constraints that are actually
20+
relevant to this environment's dependencies -- uv/edx-lint have no equivalent
21+
to pip-compile's ``-c ../constraints.txt`` cross-project chaining, so these
22+
are maintained independently from the root ones.
2023

21-
Installing the edx-sandbox environment from this file is **unsupported** and
22-
**unstable**, because the packages are not pinned.
24+
Regenerate ``uv.lock`` (and the ``base.txt`` compatibility export below) by
25+
running ``make compile-requirements`` or ``make upgrade`` from the repo root
26+
-- see the root ``Makefile``'s ``UV_SUBPROJECTS`` loop.
2327

2428
base.txt
2529
========
2630

27-
These are the latest requirement pins for edx-sandbox. They are regularly
28-
updated with the latest compatible versions of each package.
31+
A ``uv export`` of ``uv.lock``, kept for anyone still
32+
``pip install -r requirements/edx-sandbox/base.txt`` directly instead of
33+
using uv. **This is a generated file -- do not edit directly.**
2934

3035
Installing the edx-sandbox environment from this file is **supported** yet
3136
**unstable**. Breaking package upgrades and Python langugae upgrades will
@@ -36,7 +41,10 @@ releases/
3641

3742
Starting with Quince, every named Open edX release adds one of these files.
3843
They contain the requirement pins corresponding to ``base.txt`` at the time of
39-
each release.
44+
each release. These are frozen, historical snapshots -- they are not
45+
regenerated by ``make upgrade`` and are not managed by uv. Cut a new one at
46+
release time with ``uv export --project requirements/edx-sandbox --no-hashes
47+
--no-emit-project -o requirements/edx-sandbox/releases/<name>.txt``.
4048

4149
Installing the edx-sandbox environment from the *latest* release file is
4250
**supported** and **stable**. Installing the edx-sandbox environment from

requirements/edx-sandbox/base.in

Lines changed: 0 additions & 14 deletions
This file was deleted.

requirements/edx-sandbox/base.txt

Lines changed: 27 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,24 @@
1-
#
2-
# This file is autogenerated by pip-compile with Python 3.12
3-
# by the following command:
4-
#
5-
# make upgrade
6-
#
7-
cffi==2.0.0
1+
# GENERATED FILE, DO NOT EDIT DIRECTLY.
2+
# Compatibility export for anyone still 'pip install -r requirements/edx-sandbox/base.txt'
3+
# directly instead of using uv. Source of truth: requirements/edx-sandbox/pyproject.toml / uv.lock.
4+
# This file was autogenerated by uv via the following command:
5+
# uv export --frozen --no-hashes --no-emit-project
6+
cffi==2.0.0 ; platform_python_implementation != 'PyPy'
87
# via cryptography
98
chem==2.0.0
10-
# via -r requirements/edx-sandbox/base.in
11-
click==8.4.1
9+
# via edx-sandbox
10+
click==8.4.2
1211
# via nltk
1312
codejail-includes==2.0.0
14-
# via -r requirements/edx-sandbox/base.in
13+
# via edx-sandbox
14+
colorama==0.4.6 ; sys_platform == 'win32'
15+
# via
16+
# click
17+
# tqdm
1518
contourpy==1.3.3
1619
# via matplotlib
1720
cryptography==49.0.0
18-
# via -r requirements/edx-sandbox/base.in
21+
# via edx-sandbox
1922
cycler==0.12.1
2023
# via matplotlib
2124
fonttools==4.63.0
@@ -24,10 +27,9 @@ joblib==1.5.3
2427
# via nltk
2528
kiwisolver==1.5.0
2629
# via matplotlib
27-
lxml[html-clean]==5.3.2
30+
lxml==5.3.2
2831
# via
29-
# -c requirements/constraints.txt
30-
# -r requirements/edx-sandbox/base.in
32+
# edx-sandbox
3133
# lxml-html-clean
3234
# openedx-calc
3335
lxml-html-clean==0.4.4
@@ -37,52 +39,51 @@ markupsafe==3.0.3
3739
# chem
3840
# openedx-calc
3941
matplotlib==3.11.0
40-
# via -r requirements/edx-sandbox/base.in
42+
# via edx-sandbox
4143
mpmath==1.3.0
4244
# via sympy
4345
networkx==3.6.1
44-
# via -r requirements/edx-sandbox/base.in
46+
# via edx-sandbox
4547
nltk==3.9.4
4648
# via
47-
# -r requirements/edx-sandbox/base.in
4849
# chem
50+
# edx-sandbox
4951
numpy==1.26.4
5052
# via
51-
# -c requirements/constraints.txt
5253
# chem
5354
# contourpy
5455
# matplotlib
5556
# openedx-calc
5657
# scipy
5758
openedx-calc==5.0.0
58-
# via -r requirements/edx-sandbox/base.in
59+
# via edx-sandbox
5960
packaging==26.2
6061
# via matplotlib
6162
pillow==12.2.0
6263
# via matplotlib
63-
pycparser==3.0
64+
pycparser==3.0 ; implementation_name != 'PyPy' and platform_python_implementation != 'PyPy'
6465
# via cffi
6566
pyparsing==3.3.2
6667
# via
67-
# -r requirements/edx-sandbox/base.in
6868
# chem
69+
# edx-sandbox
6970
# matplotlib
7071
# openedx-calc
7172
python-dateutil==2.9.0.post0
7273
# via matplotlib
7374
random2==1.0.2
74-
# via -r requirements/edx-sandbox/base.in
75-
regex==2026.5.9
75+
# via edx-sandbox
76+
regex==2026.6.28
7677
# via nltk
7778
scipy==1.17.1
7879
# via
79-
# -r requirements/edx-sandbox/base.in
8080
# chem
81+
# edx-sandbox
8182
six==1.17.0
8283
# via python-dateutil
8384
sympy==1.14.0
8485
# via
85-
# -r requirements/edx-sandbox/base.in
86+
# edx-sandbox
8687
# openedx-calc
87-
tqdm==4.68.2
88+
tqdm==4.68.3
8889
# via nltk
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
[project]
2+
name = "edx-sandbox"
3+
version = "0.1"
4+
requires-python = ">=3.12"
5+
dependencies = [
6+
"chem", # A helper library for chemistry calculations
7+
"cryptography", # Implementations of assorted cryptography algorithms
8+
"lxml[html_clean]", # XML parser
9+
"matplotlib", # 2D plotting library
10+
"networkx", # Utilities for creating, manipulating, and studying network graphs
11+
"nltk", # Natural language processing; used by the chem package
12+
"openedx-calc",
13+
"pyparsing", # Python Parsing module
14+
"random2", # Implementation of random module that works identically under Python 2 and 3
15+
"scipy", # Math, science, and engineering library
16+
"sympy", # Symbolic math library
17+
"codejail-includes", # CodeJail manages execution of untrusted code in secure sandboxes.
18+
]
19+
20+
[tool.uv]
21+
# This isn't installed as a package itself -- it's just a locked dependency
22+
# set that gets resolved into codejail's isolated sandbox venv.
23+
package = false
24+
25+
# DO NOT EDIT constraint-dependencies DIRECTLY.
26+
# This list is managed by `edx_lint write_uv_constraints`
27+
# and will be overwritten the next time `make upgrade` is run.
28+
# - GLOBAL constraints: edit edx_lint/files/common_constraints.txt
29+
# - REPO-SPECIFIC constraints: edit [tool.edx_lint].uv_constraints in this file
30+
constraint-dependencies = [
31+
"Django<6.0",
32+
"elasticsearch<7.14.0",
33+
"numpy<2.0.0",
34+
"lxml==5.3.2",
35+
"setuptools<82",
36+
]
37+
[tool.edx_lint]
38+
# Only the subset of the root pyproject.toml's constraints that are actually
39+
# relevant to this environment's dependencies. uv/edx-lint have no equivalent
40+
# to pip-compile's "-c ../constraints.txt" cross-project chaining, so these
41+
# are independently maintained from the root ones -- see README.rst.
42+
uv_constraints = [
43+
# Needed at least until elasticsearch (used by the main edx-platform app,
44+
# not this environment) is upgraded past 7.15 -- see the root
45+
# pyproject.toml's [tool.edx_lint].uv_constraints for the full rationale.
46+
"numpy<2.0.0",
47+
# lxml and xmlsec need to be constrained because the latest version builds
48+
# against a newer version of libxml2 than what we're running with -- see
49+
# the root pyproject.toml's [tool.edx_lint].uv_constraints for the full
50+
# rationale.
51+
"lxml==5.3.2",
52+
# setuptools 82.0.0 removed pkg_resources from its distribution, but fs
53+
# (pyfilesystem2) still uses pkg_resources for namespace package
54+
# declarations -- see the root pyproject.toml's
55+
# [tool.edx_lint].uv_constraints for the full rationale.
56+
"setuptools<82",
57+
]

0 commit comments

Comments
 (0)