Skip to content

feat: comment extraction for DOCX and PPTX - #91

Merged
developer0hye merged 5 commits into
developer0hye:mainfrom
andrewjradcliffe:main
May 31, 2026
Merged

feat: comment extraction for DOCX and PPTX#91
developer0hye merged 5 commits into
developer0hye:mainfrom
andrewjradcliffe:main

Conversation

@andrewjradcliffe

Copy link
Copy Markdown
Contributor

Summary

Adds opt-in extraction of document comments for DOCX and PPTX. When enabled,
a # Comments section is appended to the end of the converted output, capturing
the commenter, the comment body, and the source the comment is attached to.

Exposed two ways:

  • Library: a new ConversionOptions.extract_comments: bool (default false).
  • CLI: a new --extract-comments flag.

The flag is a no-op for formats without comments, so it is always safe to pass.

Motivation

Comments carry review context — questions, decisions, action items — that is
otherwise lost on conversion. For LLM consumption (this crate's target), that
context is often as valuable as the body text. The feature is opt-in so default
output stays clean and existing behavior is unchanged.

Output format

The appended section uses a fixed, flat structure (global 1‑based index in order
of appearance):

# Comments

## 1
- **author**: Jane Smith (2024-01-15T09:30:00Z)
- **comment**: Please revise this paragraph.
- **source**: the quick brown fox

## 2
- **author**: Unknown
- **comment**: (reply) Agreed.
- **source**: jumped over

The section is appended to both markdown (formatted, above) and
plain_text (the same layout with all Markdown markers stripped:
Comments / 1 / author: / comment: / source:), consistent with the
crate's dual-output principle. With zero comments, the section is omitted
entirely.

Behavior

Common

  • author renders as Name (date); the date is emitted verbatim (no parsing)
    and omitted when absent (no empty parentheses). A missing author becomes
    Unknown.
  • The comment body is flattened to a single line (ASCII whitespace collapsed;
    meaningful non-ASCII spaces such as NBSP/ideographic space are preserved).
  • Replies are flattened and prefixed with (reply).
  • Malformed or unresolved comment data is skipped with a ConversionWarning
    (best-effort), so it surfaces on stderr and trips --strict.

DOCX

  • Reads word/comments.xml for author/date/body and word/commentsExtended.xml
    for reply threading.
  • The commented-on source is the text inside the commentRangeStart
    commentRangeEnd span, scanned across the body and any headers, footers,
    footnotes, and endnotes. Ordered by first anchor appearance in a fixed part
    sequence (body → headers → footers → footnotes → endnotes); comments with no
    anchor are appended last. Source is collapsed to one line and capped at 200
    characters.

PPTX

  • Supports both the legacy (commentAuthors.xml + comments/comment*.xml) and
    modern (authors.xml + comments/modernComment_*.xml, threaded) schemes.
  • PPTX comments are anchored to a point, not a text span, so the source is the
    slide label (Slide N: Title, or Slide N when the slide is untitled).

Key changes

  • src/converter/comments.rs (new): the Comment model, normalization helpers
    (collapse_ws, cap_text, format_author), and the # Comments Markdown /
    plain-text renderers (append_comments).
  • src/converter/docx.rs: comment parsing (comments.xml,
    commentsExtended.xml, range collection across content parts) and assembly,
    wired into convert_inner.
  • src/converter/pptx.rs: legacy + modern comment/author parsers and per-slide
    collection, wired into convert_inner.
  • src/converter/ooxml_utils.rs: a shared attr_value_unescaped helper that
    XML-unescapes attribute values by local name.
  • src/zip_utils.rs: read_zip_text_lossy for best-effort reads of optional
    sub-parts.
  • src/converter/mod.rs, src/lib.rs, src/runner.rs: the
    extract_comments option, the async append sites, and the CLI flag.
  • README.md, TECH_SPEC.md: documentation of the option, flag, output format,
    and per-format caveats.

convert_inner now returns the collected comments alongside the existing result
and pending-image data; the section is appended after image-placeholder
resolution in both the sync and async paths, so it never interferes with image
description.

Robustness (review hardening)

A follow-up commit addresses correctness/robustness issues found in review, each
with a regression test:

  • Self-closing text elements (<w:t/>, <p:text/>, <a:t/>) no longer leak
    stray text into comment bodies.
  • DOCX source no longer leaks past its range: range markers are honored even
    inside skipped mc:Choice branches, text is captured only inside a run, and
    each range's source is byte-bounded so a malformed unclosed range cannot
    absorb the rest of the part.
  • DOCX w14:paraId is taken only from top-level paragraphs, so a comment ending
    in a table doesn't break reply detection.
  • A PPTX slide carrying both legacy and modern comment parts no longer
    double-reports (modern is preferred).
  • A multi-line PPTX slide title no longer breaks the single-line source item.
  • Author/date attributes are XML-unescaped (R&amp;DR&D), matching body
    handling.
  • A malformed (non-UTF-8) comment sub-part degrades gracefully instead of
    aborting the whole conversion.

Testing

  • Unit tests for every parser and helper (the comment model, range collection,
    author/reply parsing, edge cases) plus end-to-end tests through the public
    convert_bytes API for DOCX and PPTX, and CLI tests for the flag.
  • Full verification: cargo test (all unit + integration suites), cargo clippy -- -D warnings, cargo fmt --check, cargo doc --no-deps (zero warnings),
    the async and async-gemini feature checks, the WASM target checks, and
    cargo build --release — all green.

Notes / limitations

  • The modern-PPTX comment schema is implemented from the MS-PPTX/ECMA-376
    specification and validated against synthetic fixtures; comment parsing is
    best-effort.
  • source truncation is on a Unicode scalar boundary (panic-safe); multi-scalar
    grapheme clusters (e.g. flag/ZWJ emoji) may be split in the truncated tail.
  • No new runtime dependencies; the feature is pure Rust and WASM-compatible.

andrewjradcliffe and others added 5 commits May 29, 2026 11:40
Add comment extraction for DOCX, gated by the new
`ConversionOptions::extract_comments` flag (default off).

- New shared `converter::comments` module: the `Comment` model plus
  pure helpers (`collapse_ws`, `cap_text`, `format_author`) and the
  `# Comments` Markdown / plain-text renderers (`append_comments`).
- DOCX parsing of `word/comments.xml` (author, date, body, paraId),
  `word/commentsExtended.xml` (reply detection), and commented-on text
  ranges (`commentRangeStart`/`End`, `commentReference`) across the body
  and any headers, footers, footnotes, and endnotes.
- Comments are ordered by first anchor appearance in a fixed part
  sequence (body, headers, footers, footnotes, endnotes); unanchored
  (orphan) comments are appended last. Replies are flattened and marked
  `(reply)`. `author` renders as `Name (date)` (date verbatim, omitted
  when absent; `Unknown` when no author). `source` is whitespace-collapsed
  and capped at 200 characters.
- `convert_inner` now returns the collected comments; the section is
  appended after image-placeholder resolution in both the sync and async
  DOCX paths.

The section is appended to both `markdown` (formatted) and `plain_text`
(markers stripped). Malformed or unresolved comment data is skipped with a
warning, per the best-effort convention.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrew Radcliffe <andrewjradcliffe@gmail.com>
Extend comment extraction (gated by `ConversionOptions::extract_comments`)
to PPTX, covering both comment schemes:

- Legacy (PowerPoint 2007–2013): `ppt/commentAuthors.xml` author registry
  and `ppt/comments/commentN.xml` files (`p:cm` with plain-text `p:text`
  body, `authorId`/`dt`).
- Modern (PowerPoint 2016+/365): `ppt/authors.xml` (GUID authors) and
  `ppt/comments/modernComment_*.xml` files (`p188:cm` with DrawingML
  `a:t` body, `authorId`/`created`); nested `p188:replyLst`/`p188:reply`
  replies are flattened, marked `(reply)`, and emitted after their parent
  in document order.

Comment parts are discovered per slide via the slide relationships and
dispatched by relationship type (the modern office/2018 namespace vs the
legacy 2006 one). Since PPTX comments are point-anchored rather than
text-anchored, `source` is the slide label (`Slide N: Title`, or `Slide N`
when untitled). `convert_inner` returns the collected comments; the
section is appended after image resolution in both the sync and async paths.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrew Radcliffe <andrewjradcliffe@gmail.com>
Expose comment extraction on the CLI via `--extract-comments`, mapped to
`ConversionOptions::extract_comments` in `build_options`. The flag is a
no-op for formats without comments.

Add public-API end-to-end tests that build DOCX and PPTX files with comment
parts in memory and assert the appended `# Comments` section via
`convert_bytes`, plus CLI tests covering `--help` listing, the CSV no-op,
and DOCX conversion under the flag.

Also applies rustfmt normalization to the comment-extraction code added in
the preceding commits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrew Radcliffe <andrewjradcliffe@gmail.com>
Document the new `extract_comments` option and `--extract-comments` flag:

- README: CLI usage example, a "Extracting Comments (DOCX / PPTX)" library
  section with example output, and a `ConversionOptions` table row.
- TECH_SPEC: the new option field, plus comment rows and notes in the DOCX
  (§4.1) and PPTX (§4.2) extraction tables.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrew Radcliffe <andrewjradcliffe@gmail.com>
Address correctness and robustness issues found in code review of the
comment-extraction feature:

- Self-closing text elements (`<w:t/>`, `<p:text/>`, `<a:t/>`) no longer
  leave the text-capture flag stuck on and leak stray text into comment
  bodies: text capture is set only on Start events (Empty has no End).
- DOCX commented-on source no longer leaks past its range: comment range
  markers are honored even inside skipped `mc:Choice` branches (so an end
  buried there still closes), text is captured only inside a run (matching
  the body parser), and each range's source is byte-bounded so a malformed
  unclosed range cannot absorb the whole part.
- DOCX `w14:paraId` is captured only from top-level paragraphs (not nested
  table cells), so reply detection is not thrown off by a trailing table.
- PPTX comment `source` (slide label) is whitespace-collapsed and capped, so
  a multi-line slide title no longer breaks the single-line list item.
- A slide carrying both legacy and modern comment parts no longer
  double-reports comments (modern is preferred).
- Author/date attributes are XML-unescaped (e.g. "R&amp;D" → "R&D"),
  matching body-text handling, via a shared `attr_value_unescaped` helper.
- `collapse_ws` collapses only ASCII whitespace, preserving meaningful
  non-ASCII spaces (NBSP, ideographic space) per the Unicode-fidelity goal.
- Comment parts are read with a lossy UTF-8 decode so a single malformed
  sub-part degrades gracefully instead of aborting the whole conversion.
- The two PPTX author-registry parsers are merged into one parameterized
  `parse_author_registry`; the `cap_text` doc is corrected to note
  scalar-level (not grapheme-level) truncation; orphan-id sort uses a
  total-order key.

Each fix has a dedicated regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrew Radcliffe <andrewjradcliffe@gmail.com>
@developer0hye
developer0hye merged commit 25a6802 into developer0hye:main May 31, 2026
7 checks passed
andrewjradcliffe pushed a commit to andrewjradcliffe/anytomd-rs that referenced this pull request Jun 1, 2026
Minor bump (1.2.2 -> 1.3.0): developer0hye#91 added opt-in comment extraction for
DOCX and PPTX — a new public `ConversionOptions.extract_comments` field
and a `--extract-comments` CLI flag — which is new, non-breaking public
API per SemVer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Yonghye Kwon <developer.0hye@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants