Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 3 additions & 15 deletions .conductor/settings.toml
Original file line number Diff line number Diff line change
@@ -1,22 +1,10 @@
[scripts]
setup = '''
if ! command -v poetry >/dev/null 2>&1; then
pipx install poetry==2.1.3
if ! command -v uv >/dev/null 2>&1; then
pipx install uv==0.12.5
fi
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated

PYTHON_BIN="$(command -v python3.11 || pyenv which python3 2>/dev/null || command -v python3)"
"$PYTHON_BIN" - <<'PY'
import sys
if not ((3, 11) <= sys.version_info < (3, 14)):
raise SystemExit(f"Python >=3.11,<3.14 is required, got {sys.version.split()[0]}")
PY

"$PYTHON_BIN" -m venv .venv
export VIRTUAL_ENV="$PWD/.venv"
export PATH="$VIRTUAL_ENV/bin:$PATH"

poetry config virtualenvs.in-project true
poetry install --with dev
uv sync --locked

if [ ! -f .env ]; then
SOURCE_ENV="$CONDUCTOR_ROOT_PATH/.env"
Expand Down
9 changes: 5 additions & 4 deletions .github/workflows/merge-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,11 @@ jobs:
with:
python-version: "3.11"

- name: Setup Poetry
uses: abatilo/actions-poetry@v3
- name: Setup uv
uses: astral-sh/setup-uv@v7
with:
poetry-version: "2.1.3"
version: "0.12.5"
enable-cache: true

- name: Build package
run: poetry build
run: uv build
24 changes: 8 additions & 16 deletions .github/workflows/pr-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,28 +18,20 @@ jobs:
with:
python-version: "3.11"

- name: Setup Poetry
uses: abatilo/actions-poetry@v3
- name: Setup uv
uses: astral-sh/setup-uv@v7
with:
poetry-version: "2.1.3"

- name: Configure Poetry
run: |
poetry config virtualenvs.create true
poetry config virtualenvs.in-project true
version: "0.12.5"
enable-cache: true

- name: Install dependencies
run: |
poetry install --with dev
run: uv sync --locked

- name: Run pyright checking
run: |
poetry run pyright pdf_craft tests
run: uv run pyright pdf_craft tests

- name: Run linting
run: |
poetry run pylint pdf_craft tests
run: uv run pylint pdf_craft tests

- name: Run tests
run: |
poetry run python test.py
run: uv run python test.py
24 changes: 10 additions & 14 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,11 @@ jobs:
with:
python-version: "3.11"

- name: Setup Poetry
uses: abatilo/actions-poetry@v3
- name: Setup uv
uses: astral-sh/setup-uv@v7
with:
poetry-version: "2.1.3"

- name: Configure Poetry
run: |
poetry config virtualenvs.create true
poetry config virtualenvs.in-project true
version: "0.12.5"
enable-cache: true

- name: Check release state
run: |
Expand All @@ -47,7 +43,7 @@ jobs:
exit 1
fi

VERSION="$(poetry version -s)"
VERSION="$(uv version --short --locked)"

if [[ ! "${VERSION}" =~ ^[0-9]+\.[0-9]+\.[0-9]+([A-Za-z0-9.-]+)?$ ]]; then
echo "pyproject.toml version must be a release version without the leading v, for example: 1.0.14"
Expand All @@ -72,19 +68,19 @@ jobs:
id: release-inputs

- name: Install dependencies
run: poetry install --with dev
run: uv sync --locked

- name: Run pyright checking
run: poetry run pyright pdf_craft tests
run: uv run pyright pdf_craft tests

- name: Run linting
run: poetry run pylint pdf_craft tests
run: uv run pylint pdf_craft tests

- name: Run tests
run: poetry run python test.py
run: uv run python test.py

- name: Build package
run: poetry build
run: uv build

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
Expand Down
7 changes: 0 additions & 7 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -108,13 +108,6 @@ ipython_config.py
# commonly ignored for libraries.
#uv.lock

# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock

# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
Expand Down
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.11
33 changes: 16 additions & 17 deletions docs/DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This guide is for human contributors. Agent-facing project routing lives in `AGE
## Requirements

- Python >= 3.11, < 3.14 (3.11.16 recommended)
- Poetry 2.x
- uv 0.12.5
- Poppler, only when running PDF rendering or conversion checks
- PyTorch, installed from the lock file through `doc-page-extractor`
- CUDA-capable PyTorch and an NVIDIA GPU, only when running real DeepSeek OCR conversion
Expand All @@ -14,19 +14,18 @@ The published `pdf-craft` package does not declare `torch` or `torchvision` dire

## Setup For Ordinary Development

Create an in-project virtual environment and install project dependencies:
Create the in-project virtual environment and install the locked project dependencies:

```shell
poetry config virtualenvs.in-project true
poetry install --with dev
uv sync --locked
```

For code reading, type checking, and the lightweight unit tests, this is usually enough.

If a task needs a specific CPU PyTorch wheel, reinstall it explicitly:

```shell
poetry run pip install --force-reinstall torch torchvision --index-url https://download.pytorch.org/whl/cpu
uv pip install --reinstall torch torchvision --default-index https://download.pytorch.org/whl/cpu
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated
```

## Setup For Real OCR Conversion
Expand All @@ -38,9 +37,9 @@ Local DeepSeek OCR requires CUDA-capable PyTorch. If the default locked wheel is
Examples:

```shell
poetry run pip install --force-reinstall torch torchvision --index-url https://download.pytorch.org/whl/cu118
poetry run pip install --force-reinstall torch torchvision --index-url https://download.pytorch.org/whl/cu121
poetry run pip install --force-reinstall torch torchvision --index-url https://download.pytorch.org/whl/cu124
uv pip install --reinstall torch torchvision --default-index https://download.pytorch.org/whl/cu118
uv pip install --reinstall torch torchvision --default-index https://download.pytorch.org/whl/cu121
uv pip install --reinstall torch torchvision --default-index https://download.pytorch.org/whl/cu124
```

Install Poppler when running PDF rendering or conversion:
Expand All @@ -57,7 +56,7 @@ brew install poppler
Verify the environment:

```shell
poetry run python -c "import torch; print(f'PyTorch version: {torch.__version__}'); print(f'CUDA available: {torch.cuda.is_available()}')"
uv run python -c "import torch; print(f'PyTorch version: {torch.__version__}'); print(f'CUDA available: {torch.cuda.is_available()}')"
pdfinfo -v
```

Expand All @@ -68,31 +67,31 @@ Vendor OCR does not require local CUDA. Copy `.env.template` to `.env`, set `PDF
The CI checks are the default validation contract:

```shell
poetry run pyright pdf_craft tests
poetry run pylint pdf_craft tests
poetry run python test.py
uv run pyright pdf_craft tests
uv run pylint pdf_craft tests
uv run python test.py
```

You can run one test module by passing the file stem or file name:

```shell
poetry run python test.py test_parser
poetry run python test.py test_parser.py
uv run python test.py test_parser
uv run python test.py test_parser.py
```

Build the package with:

```shell
poetry build
uv build
```

## Manual Conversion Checks

The scripts in `scripts/` are manual checks for conversion work. They require Poppler and an OCR configuration from `.env`. `local-deepseek` requires model downloads and CUDA; vendor modes require credentials:

```shell
poetry run python scripts/gen_md.py
poetry run python scripts/gen_epub.py
uv run python scripts/gen_md.py
uv run python scripts/gen_epub.py
```

They write conversion output under `analysing/` and use `models-cache/` for local model storage when `PDF_CRAFT_OCR_MODE=local-deepseek`.
Expand Down
33 changes: 16 additions & 17 deletions docs/DEVELOPMENT_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
## 环境要求

- Python >= 3.11, < 3.14(推荐 3.11.16)
- Poetry 2.x
- uv 0.12.5
- Poppler,仅在运行 PDF 渲染或转换检查时需要
- PyTorch,会通过 `doc-page-extractor` 从 lock file 安装
- 支持 CUDA 的 PyTorch 和 NVIDIA GPU,仅在运行真实 DeepSeek OCR 转换时需要
Expand All @@ -14,19 +14,18 @@

## 普通开发环境

创建项目内虚拟环境并安装依赖
创建项目内虚拟环境并安装锁定的项目依赖

```shell
poetry config virtualenvs.in-project true
poetry install --with dev
uv sync --locked
```

对于阅读代码、类型检查和轻量单元测试,通常这就足够了。

如果任务需要指定 CPU 版 PyTorch wheel,可以显式重装:

```shell
poetry run pip install --force-reinstall torch torchvision --index-url https://download.pytorch.org/whl/cpu
uv pip install --reinstall torch torchvision --default-index https://download.pytorch.org/whl/cpu
```

## 真实 OCR 转换环境
Expand All @@ -38,9 +37,9 @@ poetry run pip install --force-reinstall torch torchvision --index-url https://d
示例:

```shell
poetry run pip install --force-reinstall torch torchvision --index-url https://download.pytorch.org/whl/cu118
poetry run pip install --force-reinstall torch torchvision --index-url https://download.pytorch.org/whl/cu121
poetry run pip install --force-reinstall torch torchvision --index-url https://download.pytorch.org/whl/cu124
uv pip install --reinstall torch torchvision --default-index https://download.pytorch.org/whl/cu118
uv pip install --reinstall torch torchvision --default-index https://download.pytorch.org/whl/cu121
uv pip install --reinstall torch torchvision --default-index https://download.pytorch.org/whl/cu124
```

运行 PDF 渲染或转换时还需要安装 Poppler:
Expand All @@ -57,7 +56,7 @@ brew install poppler
验证环境:

```shell
poetry run python -c "import torch; print(f'PyTorch version: {torch.__version__}'); print(f'CUDA available: {torch.cuda.is_available()}')"
uv run python -c "import torch; print(f'PyTorch version: {torch.__version__}'); print(f'CUDA available: {torch.cuda.is_available()}')"
pdfinfo -v
```

Expand All @@ -68,31 +67,31 @@ pdfinfo -v
CI 检查是默认验证契约:

```shell
poetry run pyright pdf_craft tests
poetry run pylint pdf_craft tests
poetry run python test.py
uv run pyright pdf_craft tests
uv run pylint pdf_craft tests
uv run python test.py
```

可以传入文件 stem 或文件名,只运行一个测试模块:

```shell
poetry run python test.py test_parser
poetry run python test.py test_parser.py
uv run python test.py test_parser
uv run python test.py test_parser.py
```

构建发布包:

```shell
poetry build
uv build
```

## 手动转换检查

`scripts/` 中的脚本用于转换联调。它们需要 Poppler,并从 `.env` 读取 OCR 配置。`local-deepseek` 需要模型下载和 CUDA;供应商模式需要对应密钥:

```shell
poetry run python scripts/gen_md.py
poetry run python scripts/gen_epub.py
uv run python scripts/gen_md.py
uv run python scripts/gen_epub.py
```

脚本会把转换结果写入 `analysing/`。当 `PDF_CRAFT_OCR_MODE=local-deepseek` 时,脚本使用 `models-cache/` 存放本地模型。
Expand Down
4 changes: 2 additions & 2 deletions docs/RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ Configure the PyPI trusted publisher for this repository:

## 2. Prepare a release PR

For a target version such as `1.0.14`, prepare and merge a PR that includes:
For a target version such as `1.0.14`, run `uv version 1.0.14` and prepare a PR that includes:

- `pyproject.toml` version set to `1.0.14`
- `pyproject.toml` and `uv.lock` updated to `1.0.14`
- `docs/changelog/v1.0.14.md` with the release notes

Use the existing files in `docs/changelog/` as the changelog style reference.
Expand Down
Loading