Skip to content

Commit 7bc77bb

Browse files
committed
docs(examples): runnable, CI-guarded programs for each target niche
1 parent 38393cd commit 7bc77bb

8 files changed

Lines changed: 331 additions & 0 deletions

CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
159159
x86_64 musllinux wheels on every push (aarch64 builds natively at release).
160160
- Packaging metadata for the PyPI page: Trove classifiers (CPython 3.10–3.14, the
161161
supported operating systems, topics) and project URLs (Documentation, Issues).
162+
- Runnable [`examples/`](examples/) — self-contained, cross-platform programs, one
163+
per target niche (whole-tree no-orphan teardown, a readiness-gated server,
164+
supervision-until-healthy, a resource-limited sandbox). Each is exercised in CI.
162165

163166
### Changed
164167
- Renamed `Command.ok_codes()`**`success_codes()`** (clearer that it is the

README.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,11 @@ to …" tasks to working snippets — then read
170170
| [Testing your code](docs/testing.md) | The runner seam, scripted/record-replay doubles, `CliClient` |
171171
| [Platform support](docs/platforms.md) | Mechanisms, all capability matrices, every caveat |
172172

173+
Prefer whole programs to snippets? The **[`examples/`](examples/)** directory has
174+
runnable, self-contained scripts — one per niche (no-orphan teardown, a
175+
readiness-gated server, supervision, a resource-limited sandbox). Each runs on
176+
Windows, Linux, and macOS and is exercised in CI.
177+
173178
## A tour of the capabilities
174179

175180
Each section below is a taste with a pointer to its full guide.

examples/01_no_orphan_guarantee.py

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
"""No-orphan guarantee: a ProcessGroup reaps a whole tree, grandchildren included.
2+
3+
This is the core reason processkit exists. We start two children that each spawn
4+
a *grandchild*; a naive ``subprocess`` call tracks only the direct child, so the
5+
grandchildren would outlive a timeout, an exception, or a cancelled task. Inside
6+
a ``ProcessGroup``, leaving the ``with`` block tears the entire tree down in one
7+
kernel operation — a Windows Job Object, a Linux cgroup v2, or a POSIX process
8+
group.
9+
10+
Run it: python examples/01_no_orphan_guarantee.py
11+
"""
12+
13+
from __future__ import annotations
14+
15+
import sys
16+
import time
17+
18+
from processkit import Command, ProcessGroup
19+
20+
# A child that spawns a detached grandchild (a 60-second sleeper) and then sleeps
21+
# itself. Neither does any real work — they stand in for a build tool's compiler
22+
# children, a server's workers, or an agent tool's helper processes.
23+
_CHILD = (
24+
"import subprocess, sys, time; "
25+
"subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(60)']); "
26+
"time.sleep(60)"
27+
)
28+
29+
30+
def main() -> None:
31+
with ProcessGroup() as group:
32+
group.start(Command(sys.executable, ["-c", _CHILD]))
33+
group.start(Command(sys.executable, ["-c", _CHILD]))
34+
35+
# Give the children a moment to spawn their grandchildren, then look at
36+
# what the kernel container is tracking.
37+
time.sleep(0.5)
38+
members = group.members()
39+
print(f"containment mechanism : {group.mechanism}")
40+
print(f"processes in the tree : {len(members)} (PIDs {members})")
41+
print("leaving the block - the whole tree is about to be reaped...")
42+
43+
# Past this line the group is gone: both children AND their grandchildren
44+
# have been killed as a unit. No orphan survives — not even the ones we never
45+
# held a handle to.
46+
print("done - every child and grandchild has been torn down.")
47+
48+
49+
if __name__ == "__main__":
50+
main()

examples/02_wait_for_server.py

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
"""Start a server, wait until it is ready, use it, then reap the whole tree.
2+
3+
The "start a service, then talk to it" pattern — everywhere in CI orchestration
4+
and integration tests, and a constant Python pain point (racy ``sleep()`` calls,
5+
leaked server processes). Here the server runs inside a ``ProcessGroup``, so no
6+
matter how the block exits — success, exception, or timeout — the server and
7+
anything it spawned are gone.
8+
9+
``wait_for_port`` replaces the usual ``time.sleep(2) # hope it's up`` guess with
10+
an actual readiness check.
11+
12+
Run it: python examples/02_wait_for_server.py
13+
"""
14+
15+
from __future__ import annotations
16+
17+
import asyncio
18+
import socket
19+
import sys
20+
import urllib.request
21+
22+
from processkit import Command, ProcessGroup, wait_for_port
23+
24+
HOST = "127.0.0.1"
25+
26+
27+
def _free_port() -> int:
28+
"""Grab a port the OS is not using, so the example never collides with
29+
something already listening."""
30+
with socket.socket() as sock:
31+
sock.bind((HOST, 0))
32+
return int(sock.getsockname()[1])
33+
34+
35+
def _http_status(url: str) -> int:
36+
"""A blocking HTTP GET returning the status code — run off the event loop."""
37+
with urllib.request.urlopen(url, timeout=5) as response:
38+
return int(response.status)
39+
40+
41+
async def main() -> None:
42+
port = _free_port()
43+
async with ProcessGroup() as group:
44+
# Python's stdlib HTTP server stands in for your real service. Send its
45+
# logs to null: it communicates over the socket, so we don't need them —
46+
# and an undrained stdio pipe would otherwise stall a background server.
47+
server = (
48+
Command(sys.executable, ["-m", "http.server", str(port), "--bind", HOST])
49+
.stdout("null")
50+
.stderr("null")
51+
)
52+
await group.astart(server)
53+
54+
print(f"waiting for the server on {HOST}:{port} ...")
55+
await wait_for_port(HOST, port, timeout=10)
56+
print("server is accepting connections")
57+
58+
# urllib is blocking, so run it in a worker thread rather than stalling
59+
# the event loop.
60+
loop = asyncio.get_running_loop()
61+
status = await loop.run_in_executor(None, _http_status, f"http://{HOST}:{port}/")
62+
print(f"GET / -> HTTP {status}")
63+
print("leaving the block - the server tree is about to be reaped...")
64+
65+
print("done - the server has been torn down.")
66+
67+
68+
if __name__ == "__main__":
69+
asyncio.run(main())
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
"""Keep restarting a flaky worker until it comes up healthy.
2+
3+
The supervision pattern for the agent / long-lived-service niche. A worker that
4+
fails a couple of times before succeeding is restarted with exponential backoff,
5+
and a ``stop_when`` predicate ends the loop the moment a run succeeds. The
6+
restart policy, the backoff schedule, and the stop condition are all declarative
7+
— no hand-rolled retry loop.
8+
9+
Run it: python examples/03_supervise_until_healthy.py
10+
"""
11+
12+
from __future__ import annotations
13+
14+
import sys
15+
import tempfile
16+
from pathlib import Path
17+
18+
from processkit import Command, Supervisor
19+
20+
# A worker that fails its first two runs, then succeeds. It uses a file as an
21+
# attempt counter that persists across restarts — standing in for a service that
22+
# needs a dependency (a port, a migration, a mount) to become ready first.
23+
_WORKER = """
24+
import os, sys
25+
path = sys.argv[1]
26+
attempt = (int(open(path).read()) if os.path.exists(path) else 0) + 1
27+
open(path, "w").write(str(attempt))
28+
print(f"worker attempt {attempt}", flush=True)
29+
sys.exit(0 if attempt >= 3 else 1)
30+
"""
31+
32+
33+
def main() -> None:
34+
with tempfile.TemporaryDirectory() as tmp:
35+
counter = Path(tmp) / "attempts"
36+
37+
outcome = Supervisor(
38+
Command(sys.executable, ["-c", _WORKER, str(counter)]),
39+
# Restart after every run; the predicate below decides when we are
40+
# actually done (use "on_crash" to restart only on a non-zero exit).
41+
restart="always",
42+
max_restarts=5,
43+
# Small delays so the example finishes quickly; scale these up for a
44+
# real service (e.g. 0.5 / 2.0 / 30.0).
45+
backoff_initial=0.05,
46+
backoff_factor=2.0,
47+
max_backoff=1.0,
48+
stop_when=lambda result: result.is_success,
49+
).run() # or: await ...arun()
50+
51+
print(f"restarts : {outcome.restarts}") # 2 — it failed twice first
52+
print(f"stopped by : {outcome.stopped}") # 'predicate' — our stop_when fired
53+
print(f"final code : {outcome.final_result.code}")
54+
print("healthy" if outcome.final_result.is_success else "gave up")
55+
56+
57+
if __name__ == "__main__":
58+
main()
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
"""Sandbox an untrusted child tree with kernel-enforced resource limits.
2+
3+
The differentiator versus a plain subprocess wrapper: a ``ProcessGroup`` can cap
4+
a whole tree's memory, process count, and CPU — enforced by the Windows Job
5+
Object or a Linux cgroup v2. We also lock the command itself down (empty
6+
environment, bounded captured output, die-with-parent) so a misbehaving tool
7+
cannot run away with the machine.
8+
9+
Kernel resource limits need privileges the environment may not grant: inside a
10+
container, a systemd user session, or a non-root cgroup the kernel forbids them,
11+
and macOS (a POSIX process group) has no equivalent. processkit is honest about
12+
this — it raises rather than silently ignoring the cap — so this example catches
13+
that and degrades to "contained, but uncapped", staying runnable anywhere.
14+
15+
Run it: python examples/04_sandbox_resource_limits.py
16+
"""
17+
18+
from __future__ import annotations
19+
20+
import sys
21+
22+
from processkit import Command, ProcessGroup, ResourceLimit, Unsupported
23+
24+
_MiB = 1024 * 1024
25+
26+
# A short-lived stand-in for an untrusted tool doing a little work.
27+
_TOOL = "import time; time.sleep(0.1)"
28+
29+
30+
def _locked_down_tool() -> Command:
31+
"""An untrusted command tied down independently of the group's limits: an
32+
empty environment (only PATH allow-listed), a bounded captured output, and
33+
kill-on-parent-death so it cannot outlive us even without explicit teardown."""
34+
return (
35+
Command(sys.executable, ["-c", _TOOL])
36+
.env_clear()
37+
.inherit_env(["PATH"])
38+
.kill_on_parent_death()
39+
.output_limit(max_bytes=8 * _MiB)
40+
)
41+
42+
43+
def _run(
44+
*,
45+
max_memory: int | None = None,
46+
max_processes: int | None = None,
47+
cpu_quota: float | None = None,
48+
) -> None:
49+
with ProcessGroup(
50+
max_memory=max_memory,
51+
max_processes=max_processes,
52+
cpu_quota=cpu_quota,
53+
) as group:
54+
group.start(_locked_down_tool())
55+
print(f" mechanism : {group.mechanism}")
56+
# Live usage stats are a bonus, not the point — some mechanisms (a POSIX
57+
# process group) can't report them, so don't let that mask the sandbox.
58+
try:
59+
stats = group.stats()
60+
print(f" active processes : {stats.active_process_count}")
61+
print(f" peak memory (bytes) : {stats.peak_memory_bytes}")
62+
except Unsupported:
63+
print(" usage stats : unavailable on this platform")
64+
65+
66+
def main() -> None:
67+
try:
68+
_run(max_memory=512 * _MiB, max_processes=64, cpu_quota=1.0)
69+
print("ran the tool under kernel-enforced memory / process / CPU limits.")
70+
except (ResourceLimit, Unsupported) as exc:
71+
print(f"kernel resource limits are not permitted here: {exc}")
72+
print("(typical in containers / non-root cgroups / macOS) - running uncapped.")
73+
_run()
74+
print("ran the tool contained, but without resource caps.")
75+
76+
77+
if __name__ == "__main__":
78+
main()

examples/README.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# processkit examples
2+
3+
Runnable, self-contained programs — each maps to one of the niches processkit is
4+
built for. They use only the standard library plus `processkit`, spawn their own
5+
child processes (via the running Python), and work the same on Windows, Linux,
6+
and macOS. Every one is exercised in CI, so they stay current with the API.
7+
8+
Run any of them from the repository root:
9+
10+
```bash
11+
python examples/01_no_orphan_guarantee.py
12+
```
13+
14+
| Example | Shows | Niche |
15+
|---|---|---|
16+
| [`01_no_orphan_guarantee.py`](01_no_orphan_guarantee.py) | A `ProcessGroup` reaps a whole child→grandchild tree on block exit | The core guarantee |
17+
| [`02_wait_for_server.py`](02_wait_for_server.py) | Start a server, `await wait_for_port(...)`, make a request, tear the tree down (async) | CI orchestration / integration tests |
18+
| [`03_supervise_until_healthy.py`](03_supervise_until_healthy.py) | `Supervisor` with restart + backoff + a `stop_when` predicate | Agents / long-lived services |
19+
| [`04_sandbox_resource_limits.py`](04_sandbox_resource_limits.py) | Memory / process / CPU caps on a locked-down untrusted child | Sandboxing untrusted tools |
20+
21+
For task-sized snippets rather than whole programs, see the
22+
[cookbook](../docs/cookbook.md); for the full treatment of any area, the
23+
[guide set](../docs/README.md).

tests/test_examples.py

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
"""Every script under examples/ must run to a clean exit.
2+
3+
These are the programs users copy first, so a broken one is a broken first
4+
impression — and because they live outside the package, nothing else would catch
5+
API drift in them. Each runs in a child interpreter (they spawn processes and
6+
bind ports of their own), so this mirrors exactly what `python examples/<name>.py`
7+
does for a user.
8+
"""
9+
10+
from __future__ import annotations
11+
12+
import subprocess
13+
import sys
14+
from pathlib import Path
15+
16+
import pytest
17+
18+
_EXAMPLES_DIR = Path(__file__).resolve().parent.parent / "examples"
19+
_EXAMPLES = sorted(_EXAMPLES_DIR.glob("*.py"))
20+
21+
22+
def test_examples_directory_is_populated() -> None:
23+
# Guard against a false green if the directory moves or the glob breaks:
24+
# an empty parametrization would otherwise report zero tests, silently.
25+
assert _EXAMPLES, f"no example scripts found under {_EXAMPLES_DIR}"
26+
27+
28+
@pytest.mark.parametrize("script", _EXAMPLES, ids=lambda p: p.name)
29+
def test_example_runs_cleanly(script: Path) -> None:
30+
result = subprocess.run(
31+
[sys.executable, str(script)],
32+
capture_output=True,
33+
text=True,
34+
timeout=60,
35+
check=False,
36+
)
37+
assert result.returncode == 0, (
38+
f"{script.name} exited {result.returncode}\n"
39+
f"--- stdout ---\n{result.stdout}\n--- stderr ---\n{result.stderr}"
40+
)
41+
# A caught-and-handled error is fine (e.g. the sandbox example degrades on a
42+
# ResourceLimit); an *un*caught one surfaces as a traceback on stderr.
43+
assert "Traceback (most recent call last)" not in result.stderr, (
44+
f"{script.name} raised an uncaught exception:\n{result.stderr}"
45+
)

0 commit comments

Comments
 (0)