Skip to content

Commit 36dc1e5

Browse files
caio-pizzolcaio-pizzol
committed
Merge remote-tracking branch 'origin/main' into gabriel/sd-2447-bug-toc-renders-with-incorrect-formatting-after-document
# Conflicts: # packages/super-editor/src/editors/v1/core/super-converter/v3/handlers/sd/tableOfContents/tableOfContents-translator.js # packages/super-editor/src/editors/v1/core/super-converter/v3/handlers/sd/tableOfContents/tableOfContents-translator.test.js
2 parents 1d776c6 + ff220a9 commit 36dc1e5

477 files changed

Lines changed: 132459 additions & 107902 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/scripts/package-lock.json‎

Lines changed: 118 additions & 307 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎.github/workflows/ci-coverage.yml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
name: Coverage
2+
3+
permissions:
4+
contents: read
5+
6+
on:
7+
push:
8+
branches: [main]
9+
paths:
10+
- 'packages/superdoc/src/**'
11+
- 'packages/superdoc/vite.config.js'
12+
workflow_dispatch:
13+
14+
concurrency:
15+
group: coverage-${{ github.sha }}
16+
cancel-in-progress: true
17+
18+
jobs:
19+
coverage:
20+
runs-on: ubuntu-latest
21+
steps:
22+
- uses: actions/checkout@v6
23+
24+
- uses: pnpm/action-setup@v4
25+
26+
- uses: actions/setup-node@v6
27+
with:
28+
node-version-file: .nvmrc
29+
cache: pnpm
30+
31+
- run: pnpm install
32+
33+
- run: pnpm --filter @superdoc-dev/superdoc-yjs-collaboration build
34+
35+
- run: pnpm --filter superdoc exec vitest run --coverage
36+
37+
- uses: codecov/codecov-action@v5
38+
with:
39+
token: ${{ secrets.CODECOV_TOKEN }}
40+
files: packages/superdoc/coverage/lcov.info
41+
flags: superdoc

‎.github/workflows/ci-superdoc.yml‎

Lines changed: 40 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ jobs:
4040

4141
- uses: oven-sh/setup-bun@v2
4242
with:
43-
bun-version: 1.3.11
43+
bun-version: 1.3.12
4444

4545
- name: Install canvas system dependencies
4646
run: |
@@ -72,6 +72,13 @@ jobs:
7272
- name: Typecheck
7373
run: pnpm run type-check
7474

75+
- name: Consumer typecheck
76+
run: |
77+
cd packages/superdoc && pnpm pack --pack-destination .
78+
cd ../../tests/consumer-typecheck
79+
npm install ../../packages/superdoc/superdoc-*.tgz --no-save
80+
npx tsc --noEmit
81+
7582
unit-tests:
7683
needs: build
7784
runs-on: ubuntu-latest
@@ -101,7 +108,7 @@ jobs:
101108

102109
- uses: oven-sh/setup-bun@v2
103110
with:
104-
bun-version: 1.3.11
111+
bun-version: 1.3.12
105112

106113
- name: Install canvas system dependencies
107114
run: |
@@ -164,13 +171,13 @@ jobs:
164171
if: matrix.name == 'other-packages'
165172
run: pnpm test:slow
166173

167-
- name: Install Playwright for UMD smoke test
174+
- name: Install Playwright for CDN smoke test
168175
if: matrix.name == 'other-packages'
169-
run: pnpm --filter @superdoc/umd-smoke-test exec playwright install --with-deps chromium
176+
run: pnpm --filter @superdoc/cdn-smoke-test exec playwright install --with-deps chromium
170177

171-
- name: Run UMD smoke test
178+
- name: Run CDN smoke test
172179
if: matrix.name == 'other-packages'
173-
working-directory: packages/superdoc/tests/umd-smoke
180+
working-directory: packages/superdoc/tests/cdn-smoke
174181
run: pnpm test
175182

176183
cli-tests:
@@ -188,7 +195,7 @@ jobs:
188195

189196
- uses: oven-sh/setup-bun@v2
190197
with:
191-
bun-version: 1.3.11
198+
bun-version: 1.3.12
192199

193200
- name: Install dependencies
194201
run: pnpm install
@@ -199,9 +206,34 @@ jobs:
199206
- name: Run CLI tests
200207
run: pnpm run test:cli
201208

209+
coverage:
210+
needs: build
211+
runs-on: ubuntu-latest
212+
steps:
213+
- uses: actions/checkout@v6
214+
215+
- uses: pnpm/action-setup@v4
216+
217+
- uses: actions/setup-node@v6
218+
with:
219+
node-version-file: .nvmrc
220+
cache: pnpm
221+
222+
- run: pnpm install
223+
224+
- run: pnpm --filter @superdoc-dev/superdoc-yjs-collaboration build
225+
226+
- run: pnpm --filter superdoc exec vitest run --coverage
227+
228+
- uses: codecov/codecov-action@v5
229+
with:
230+
token: ${{ secrets.CODECOV_TOKEN }}
231+
files: packages/superdoc/coverage/lcov.info
232+
flags: superdoc
233+
202234
validate:
203235
if: always()
204-
needs: [build, unit-tests, cli-tests]
236+
needs: [build, unit-tests, cli-tests, coverage]
205237
runs-on: ubuntu-latest
206238
steps:
207239
- name: Check results

‎.github/workflows/release-cli.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,8 @@ jobs:
6363
registry-url: 'https://registry.npmjs.org'
6464

6565
- uses: oven-sh/setup-bun@v2
66+
with:
67+
bun-version: 1.3.12
6668

6769
- name: Cache apt packages
6870
uses: actions/cache@v5

‎.github/workflows/release-sdk.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,8 @@ jobs:
8888
registry-url: "https://registry.npmjs.org"
8989

9090
- uses: oven-sh/setup-bun@v2
91+
with:
92+
bun-version: 1.3.12
9193

9294
- uses: actions/setup-python@v5
9395
with:
@@ -234,6 +236,8 @@ jobs:
234236
registry-url: "https://registry.npmjs.org"
235237

236238
- uses: oven-sh/setup-bun@v2
239+
with:
240+
bun-version: 1.3.12
237241

238242
- uses: actions/setup-python@v5
239243
with:

‎.github/workflows/release-superdoc.yml‎

Lines changed: 58 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# Auto-releases on push to:
22
# - main (@next channel)
33
# - stable (@latest channel)
4+
# Manual PR preview: dispatch with pr_number to publish @pr-<number>
45
name: 📦 Release superdoc
56

67
on:
@@ -18,10 +19,16 @@ on:
1819
- 'pnpm-workspace.yaml'
1920
- '!**/*.md'
2021
workflow_dispatch:
22+
inputs:
23+
pr_number:
24+
description: 'PR number to publish a preview package for (leave empty for normal release)'
25+
required: false
26+
type: number
2127

2228
permissions:
2329
contents: write
2430
packages: write
31+
pull-requests: write
2532

2633
concurrency:
2734
group: ${{ github.ref_name == 'stable' && 'release-stable-writeback' || format('{0}-{1}', github.workflow, github.ref) }}
@@ -38,13 +45,25 @@ jobs:
3845
app-id: ${{ secrets.APP_ID }}
3946
private-key: ${{ secrets.APP_PRIVATE_KEY }}
4047

48+
# PR preview: check out the PR branch
49+
- name: Get PR head ref
50+
if: inputs.pr_number
51+
id: pr
52+
env:
53+
GH_TOKEN: ${{ github.token }}
54+
run: |
55+
PR_DATA=$(gh api repos/${{ github.repository }}/pulls/${{ inputs.pr_number }} --jq '{ref: .head.ref, sha: .head.sha}')
56+
echo "ref=$(echo $PR_DATA | jq -r .ref)" >> $GITHUB_OUTPUT
57+
echo "sha=$(echo $PR_DATA | jq -r .sha)" >> $GITHUB_OUTPUT
58+
4159
- uses: actions/checkout@v6
4260
with:
4361
fetch-depth: 0
62+
ref: ${{ steps.pr.outputs.sha || '' }}
4463
token: ${{ steps.generate_token.outputs.token }}
4564

4665
- name: Refresh stable branch head
47-
if: github.ref_name == 'stable'
66+
if: github.ref_name == 'stable' && !inputs.pr_number
4867
run: |
4968
git fetch origin "${{ github.ref_name }}" --tags
5069
git checkout -B "${{ github.ref_name }}" "origin/${{ github.ref_name }}"
@@ -81,10 +100,48 @@ jobs:
81100
- name: Install dependencies
82101
run: pnpm install
83102

103+
# PR preview: set version before build so it's baked into the package
104+
- name: Set preview version
105+
if: inputs.pr_number
106+
id: version
107+
run: |
108+
BASE_VERSION=$(node -p "require('./packages/superdoc/package.json').version")
109+
PREVIEW_VERSION="${BASE_VERSION}-pr.${{ inputs.pr_number }}.$(date +%s)"
110+
echo "version=$PREVIEW_VERSION" >> $GITHUB_OUTPUT
111+
cd packages/superdoc
112+
npm version "$PREVIEW_VERSION" --no-git-tag-version
113+
84114
- name: Build packages
85115
run: pnpm run build
86116

117+
# PR preview: publish with pr-<number> dist-tag
118+
- name: Publish PR preview
119+
if: inputs.pr_number
120+
env:
121+
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
122+
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
123+
run: |
124+
node scripts/publish-superdoc.cjs \
125+
--dist-tag pr-${{ inputs.pr_number }} \
126+
--skip-build
127+
128+
- name: Comment on PR
129+
if: inputs.pr_number
130+
env:
131+
GH_TOKEN: ${{ github.token }}
132+
run: |
133+
gh pr comment ${{ inputs.pr_number }} --body "$(cat <<'EOF'
134+
📦 **Preview published:** `superdoc@${{ steps.version.outputs.version }}`
135+
136+
```bash
137+
npm install superdoc@pr-${{ inputs.pr_number }}
138+
```
139+
EOF
140+
)"
141+
142+
# Normal release: semantic-release (only when NOT a PR preview)
87143
- name: Release
144+
if: ${{ !inputs.pr_number }}
88145
env:
89146
GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }}
90147
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

‎.github/workflows/update-contributors.yml‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -120,8 +120,10 @@ jobs:
120120
git commit -m "docs: add $AUTHOR to community contributors"
121121
git push origin "$BRANCH"
122122
123-
gh pr create \
123+
PR_URL=$(gh pr create \
124124
--base main \
125125
--head "$BRANCH" \
126126
--title "docs: add $AUTHOR to community contributors" \
127-
--body "Auto-generated: adds $AUTHOR to the README community contributors section."
127+
--body "Auto-generated: adds $AUTHOR to the README community contributors section.")
128+
129+
gh pr merge "$PR_URL" --squash --auto

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,3 +108,4 @@ packages/sdk/tools/*.json
108108
# deploys directly from git with no pre-build hook support.
109109

110110
.playwright-cli/
111+
.prqrc.json

‎AGENTS.md‎

Lines changed: 51 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -118,22 +118,70 @@ Many packages use `.js` files with JSDoc `@typedef` for type definitions (e.g.,
118118

119119
## AI Eval Suite
120120

121-
The `evals/` directory contains a Promptfoo-based evaluation suite for validating AI tool call quality.
121+
The `evals/` directory contains a Promptfoo-based evaluation suite with three levels of evaluation.
122+
123+
### Level 1: Deterministic Evals (tool selection + argument accuracy)
122124

123125
| Command | What it does | Cost |
124126
|---------|-------------|------|
125127
| `pnpm --filter @superdoc-testing/evals run eval` | Run deterministic evals (reading + argument tests) | ~$0.30 |
126128
| `pnpm --filter @superdoc-testing/evals run eval:reading` | Run reading tool tests only | ~$0.15 |
127-
| `pnpm --filter @superdoc-testing/evals run eval:gdpval` | Run GDPval benchmark (Model+SuperDoc vs Model-Only) | ~$1-2 |
128129
| `pnpm --filter @superdoc-testing/evals run eval:view` | Open Promptfoo web UI with results | Free |
129130
| `pnpm --filter @superdoc-testing/evals run baseline:save <label>` | Save versioned results snapshot | Free |
130131

131132
Tool definitions are extracted from `packages/sdk/tools/` via `evals/tools/extract.mjs`. Run `pnpm run generate:all` first if SDK artifacts are missing.
132133

133-
Test files are YAML in `evals/tests/`. Each test has a `vars.task` prompt and JavaScript assertions that check tool call structure (Level 1: tool selection + argument accuracy, not execution).
134+
Test files are YAML in `evals/tests/`. Each test has a `vars.task` prompt and JavaScript assertions that check tool call structure (tool selection + argument accuracy, not execution).
134135

135136
The system prompt at `evals/prompts/agent.txt` is a copy of the proven prompt from `examples/eval-demo/lib/agent.ts`. Update both when changing the prompt.
136137

138+
### Level 2: GDPval Benchmark (Model+SuperDoc vs Model-Only)
139+
140+
| Command | What it does | Cost |
141+
|---------|-------------|------|
142+
| `pnpm --filter @superdoc-testing/evals run eval:gdpval` | Run GDPval benchmark | ~$1-2 |
143+
144+
### Level 3: DOCX Agent Benchmark (real agents, real documents)
145+
146+
Runs actual Claude Code and Codex CLIs against DOCX tasks, comparing their performance with and without SuperDoc tools. 4 conditions x 2 agents x N tasks.
147+
148+
**Conditions:**
149+
150+
| Condition | What the agent gets |
151+
|-----------|-------------------|
152+
| baseline | No skill, agent figures out DOCX on its own |
153+
| baseline-with-docx-skill | Anthropic's DOCX skill (unzip + XML editing) |
154+
| superdoc-mcp | SuperDoc MCP server (`superdoc_open`, `superdoc_get_content`, etc.) |
155+
| superdoc-cli | SuperDoc CLI on PATH |
156+
157+
**Tasks:** 3 reading (extract headings, entity names, financial figures) + 3 editing (replace entity name, insert section, fill placeholders).
158+
159+
**Metrics per task:** correctness (pass/fail), collateral (no unintended changes), steps (agent turn count), latency (seconds), tokens (input + output), path (which DOCX approach was used).
160+
161+
| Command | What it does | Cost |
162+
|---------|-------------|------|
163+
| `pnpm --filter @superdoc-testing/evals run eval:benchmark` | Run full benchmark | ~15 min |
164+
| `pnpm --filter @superdoc-testing/evals run eval:benchmark:codex` | Run Codex conditions only | ~8 min |
165+
| `pnpm --filter @superdoc-testing/evals run eval:benchmark:claude` | Run Claude Code conditions only | ~8 min |
166+
| `pnpm --filter @superdoc-testing/evals run eval:benchmark:report` | Generate comparison report (Markdown + CSV) | Free |
167+
168+
**Prerequisites:**
169+
- `OPENAI_API_KEY` in `evals/.env` (for Codex; use `codex login --with-api-key` for API key auth)
170+
- Claude Code installed locally (uses local auth, no API key needed in `.env`)
171+
- MCP server built: `cd apps/mcp && pnpm run build`
172+
- CLI built: check `apps/cli/dist/index.js` exists
173+
174+
**Key files:**
175+
176+
| File | Purpose |
177+
|------|---------|
178+
| `evals/config/benchmark.promptfoo.yaml` | Level 3 Promptfoo config (8 providers) |
179+
| `evals/suites/benchmark/tests/agent-benchmark-v2.yaml` | Benchmark tasks with assertions |
180+
| `evals/providers/claude-code-agent.mjs` | Claude Agent SDK provider |
181+
| `evals/providers/codex-agent.mjs` | Codex SDK provider |
182+
| `evals/suites/benchmark/reports/benchmark-report.mjs` | Markdown + CSV report generator |
183+
| `evals/fixtures/vendor/vendor-docx-skill.md` | Anthropic's DOCX skill for baseline-with-docx-skill condition |
184+
137185
## Generated Artifacts
138186

139187
These directories are produced by `pnpm run generate:all`:

‎README.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
<div align="center">
1212
<a href="https://www.npmjs.com/package/superdoc" target="_blank"><img src="https://img.shields.io/npm/v/superdoc.svg?color=1355ff" height="22px"></a>
1313
<a href="https://www.npmjs.com/package/superdoc" target="_blank"><img src="https://img.shields.io/npm/dm/superdoc.svg?color=1355ff" height="22px"></a>
14+
<a href="https://codecov.io/gh/superdoc-dev/superdoc" target="_blank"><img src="https://codecov.io/gh/superdoc-dev/superdoc/branch/main/graph/badge.svg" height="22px"></a>
1415
<a href="https://www.gnu.org/licenses/agpl-3.0" target="_blank"><img src="https://img.shields.io/badge/License-AGPL%20v3-1355ff.svg?color=1355ff" height="22px"></a>
1516
<a href="https://github.com/superdoc-dev/superdoc" target="_blank"><img src="https://img.shields.io/github/stars/superdoc-dev/superdoc?style=flat&color=1355ff" height="22px"></a>
1617
<a href="https://discord.com/invite/b9UuaZRyaB" target="_blank"><img src="https://img.shields.io/badge/discord-join-1355ff" height="22px"></a>
@@ -159,6 +160,14 @@ Special thanks to these community members who have contributed code to SuperDoc:
159160
<a href="https://github.com/Branc0"><img src="https://github.com/Branc0.png" width="50" height="50" alt="Branc0" title="Rafael Rocha de Azevedo" /></a>
160161
<a href="https://github.com/Muhammad-Nur-Alamsyah-Anwar"><img src="https://github.com/Muhammad-Nur-Alamsyah-Anwar.png" width="50" height="50" alt="Muhammad-Nur-Alamsyah-Anwar" title="Alam" /></a>
161162
<a href="https://github.com/andrewsrigom"><img src="https://github.com/andrewsrigom.png" width="50" height="50" alt="andrewsrigom" title="Andrews Rigom" /></a>
163+
<a href="https://github.com/iguit0"><img src="https://github.com/iguit0.png" width="50" height="50" alt="iguit0" title="Igor Alves" /></a>
164+
<a href="https://github.com/PeterHollens"><img src="https://github.com/PeterHollens.png" width="50" height="50" alt="PeterHollens" title="Peter Hollens" /></a>
165+
<a href="https://github.com/baristaGeek"><img src="https://github.com/baristaGeek.png" width="50" height="50" alt="baristaGeek" title="Esteban Vargas" /></a>
166+
<a href="https://github.com/Anuj52"><img src="https://github.com/Anuj52.png" width="50" height="50" alt="Anuj52" title="Anuj Chaudhary" /></a>
167+
<a href="https://github.com/Abdeltoto"><img src="https://github.com/Abdeltoto.png" width="50" height="50" alt="Abdeltoto" title="Abdel ATIA" /></a>
168+
<a href="https://github.com/JoaaoVerona"><img src="https://github.com/JoaaoVerona.png" width="50" height="50" alt="JoaaoVerona" title="João Vitor Verona Biazibetti" /></a>
169+
<a href="https://github.com/michaelreavant"><img src="https://github.com/michaelreavant.png" width="50" height="50" alt="michaelreavant" title="michaelreavant" /></a>
170+
<a href="https://github.com/ArturQuirino"><img src="https://github.com/ArturQuirino.png" width="50" height="50" alt="ArturQuirino" title="ARTUR QUIRINO" /></a>
162171

163172
Want to see your avatar here? Check the [Contributing Guide](CONTRIBUTING.md) to get started.
164173

0 commit comments

Comments
 (0)