Add test case and improve skill.md

Author: mathpluscode

.claude-plugin/marketplace.json

@@ -16,7 +16,7 @@
         "path": "./"
       },
       "description": "A bibliography toolkit for LaTeX",
-      "version": "1.5.0",
+      "version": "1.5.1",
       "keywords": ["bibtex", "bibliography", "latex", "overleaf", "academic", "reference", "citation"],
       "category": "academic",
       "license": "MIT"

.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "bibtools",
   "description": "A bibliography toolkit for LaTeX",
-  "version": "1.5.0",
+  "version": "1.5.1",
   "author": {
     "name": "Yunguan Fu"
   },

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "bibtools"
-version = "1.5.0"
+version = "1.5.1"
 description = "A bibliography toolkit for LaTeX, built as agent skills"
 requires-python = ">=3.10"
 license = "MIT"

skills/bibtidy/SKILL.md

@@ -3,16 +3,18 @@ name: bibtidy
 description: Use when the user wants to validate, check, or fix a BibTeX (.bib) reference file — wrong authors, stale arXiv preprints, incorrect metadata, duplicate entries, formatting issues
 metadata:
   short-description: Validate and fix BibTeX reference files
-argument-hint: <path-to-file.bib>
-allowed-tools: Bash(python3 *), Read, Edit, Agent, WebSearch
+allowed-tools: Bash(python3 *), Bash(cp *), Bash(rm *), Read, Agent, WebSearch
 ---
 
 Use this skill whenever the user wants to validate, check, clean up, or fix a BibTeX `.bib` file.
 
-If invoked with an argument, operate on this file: `$ARGUMENTS`.
+If the user request or skill invocation includes an explicit `.bib` path, operate on that file.
 
-If the file path is missing or does not exist, ask for the path. In Claude Code slash-command usage, the fallback is:
-"Usage: /bibtidy <path-to-file.bib>"
+If the file path is missing or does not exist, ask for the path.
+
+Usage examples:
+- Claude Code slash command: `/bibtidy <path-to-file.bib>`
+- Codex request: `Use the bibtidy skill to validate and fix <path-to-file.bib>`
 
 You are a meticulous academic reference checker. Process the .bib file entry by entry, verifying each against external sources and fixing errors in-place.
 
@@ -48,7 +50,7 @@ Use `$TOOLS_DIR` in every invocation.
 
 ## Output Format for Changed Entries
 
-Each targeted edit MUST contain the original entry, one or more source URLs, an explanation, and the corrected entry. Include URLs for all sources used (CrossRef, DOI, venue page), each on its own `% bibtidy:` line.
+For `fix` patches, each targeted edit MUST contain the original entry, one or more source URLs, an explanation, and the corrected entry. Include URLs for all sources used (CrossRef, DOI, venue page), each on its own `% bibtidy:` line.
 
 ```
 % @<type>{<key>,
@@ -70,6 +72,10 @@ Each targeted edit MUST contain the original entry, one or more source URLs, an
 - **Part 3** — `% bibtidy: ` followed by explanation of what changed.
 - **Part 4** — corrected entry.
 
+Exceptions:
+- `not_found` entries get the `% bibtidy: NOT FOUND ...` line plus the fully commented-out original entry. Do NOT add a URL line.
+- `duplicate` entries get `% bibtidy: DUPLICATE of <other_key> — consider removing` above the original entry. Do NOT add URL or explanation lines unless the tool behavior changes.
+
 For unchanged entries, do NOT add any comments or URLs.
 
 ## Workflow
@@ -79,16 +85,16 @@ For unchanged entries, do NOT add any comments or URLs.
 3. Preserve `@string`, `@preamble`, `@comment` blocks verbatim
 4. Run duplicate detection: `python3 $TOOLS_DIR/duplicates.py <file.bib>`
 5. **Run field comparison**: `python3 $TOOLS_DIR/compare.py <file.bib>` — this programmatically compares every entry against CrossRef and returns exact field-level mismatches. Do NOT skip this step or rely on visual comparison alone. The output is a JSON list; each element has `key`, `versions` (a list of CrossRef matches, each with `mismatches`, `url`, `doi`, etc.), and `error`. **Skip rule**: if an entry has zero mismatches across all versions and no error in the compare.py output, skip it entirely — do NOT investigate, modify, or add comments to it. Only proceed with entries that compare.py flagged (mismatches, errors, or duplicates from step 4).
-6. **Verify every planned modification with web search** — for entries that compare.py flagged with mismatches or errors, and for entries flagged as duplicates, gather a source URL and double-check the modification via web search. Entries where `compare.py` returned an error (e.g. "No exact title match") still need full verification — the subagent should search for the paper and check all fields. **Important: subagents MUST NOT override `compare.py` field values.** CrossRef is the authoritative source for metadata (pages, volume, number, etc.) because it receives data directly from publishers via DOI registration. When web search finds a conflicting value (e.g. different page numbers on a conference website), always use the CrossRef value and add `% bibtidy: REVIEW` if desired — but do NOT keep the old value.
+6. **Verify every planned modification with web search** — for entries that compare.py flagged with mismatches or errors, and for entries flagged as duplicates, verify the planned action via web search. For `fix` patches, gather one or more source URLs. Entries where `compare.py` returned an error (e.g. "No exact title match") still need full verification — the verification agent should search for the paper and check all fields. **Important: verification agents MUST NOT override `compare.py` field values.** CrossRef is the authoritative source for metadata (pages, volume, number, etc.) because it receives data directly from publishers via DOI registration. When web search finds a conflicting value (e.g. different page numbers on a conference website), always use the CrossRef value and add `% bibtidy: REVIEW` if desired — but do NOT keep the old value.
 7. **Flag hallucinated/non-existent references** — if compare.py returned an error (e.g. "No CrossRef results found" or "No exact title match in CrossRef results") AND web search also finds no matching paper, the reference likely does not exist. Add `% bibtidy: NOT FOUND — no matching paper on CrossRef or web search; verify this reference exists` above the entry, then comment out the entire entry (prefix every line with `% `). Do NOT add a URL line.
-8. Apply fixes **sequentially** with targeted edits — do NOT rewrite the entire file. You MUST apply **every** mismatch reported by `compare.py` — do not skip any field (including `number`, `pages`, `volume`). Use the `crossref_value` exactly as given (do NOT rephrase, reformat, or partially apply it). For title mismatches on preprint→published upgrades, replace the entire title with the CrossRef title — do NOT try to edit parts of the old title. Never reject a CrossRef value because another source disagrees. **URL rule**: for every changed entry, include `% bibtidy: <url>` lines so the user can verify the correction. Include the CrossRef URL from compare.py's `url` field when available, plus any other authoritative source (DOI URL, venue page) found via web search.
+8. Apply fixes **sequentially** using `edit.py` — do NOT edit the .bib file directly with agent editing tools (for example, Claude Code Edit or Codex `apply_patch`), and do NOT rewrite the entire file. Build a patches.json for each entry (or batch) and run `python3 $TOOLS_DIR/edit.py <file.bib> <patches.json>`. This ensures the commented original, source URLs, and explanation are always included. You MUST apply **every** mismatch reported by `compare.py` — do not skip any field (including `number`, `pages`, `volume`). Use the `crossref_value` exactly as given (do NOT rephrase, reformat, or partially apply it). For title mismatches on preprint→published upgrades, replace the entire title with the CrossRef title — do NOT try to edit parts of the old title. Never reject a CrossRef value because another source disagrees. Every patch MUST include `urls` (list of source URLs) and `explanation` (what changed and why). Include the CrossRef URL from compare.py's `url` field when available, plus any other authoritative source (DOI URL, venue page) found via web search.
 9. Run format validation; fix violations and re-run until clean
 10. Delete backup: `rm <file>.bib.orig`
 11. Print a Markdown summary table with headers `Metric | Count` and exactly these rows: total entries, verified, fixed, not found. Do NOT include a separate "needs manual review" row.
 
 ## Parallel Verification with Subagents
 
-Use subagents, when available, to verify multiple entries concurrently. This dramatically reduces wall-clock time (e.g., 7 entries: ~1 min parallel vs ~5 min sequential; 100 entries: ~3 min vs ~40 min).
+Use subagents, when available, to verify multiple entries concurrently. This dramatically reduces wall-clock time (e.g., 7 entries: ~1 min parallel vs ~5 min sequential; 100 entries: ~3 min vs ~40 min). If subagents are unavailable, do the same verification work sequentially yourself.
 
 **Step 1 — Dispatch verification agents:** For entries that `compare.py` flagged with mismatches or errors, and any duplicate entries you plan to annotate, launch a subagent that:
 - For mismatches: uses web search to confirm the CrossRef data (especially for preprint upgrades and author changes)
@@ -97,11 +103,11 @@ Use subagents, when available, to verify multiple entries concurrently. This dra
 
 **When CrossRef fails**, find the paper's official venue page via web search. Many venues (JMLR, NeurIPS, CVPR, etc.) provide a downloadable `.bib` file — fetch it directly when possible. An official `.bib` is the most reliable source: it has exact title, authors, volume, number, and pages with no guessing.
 
-Launch all agents in a single message so they run concurrently. Group into batches of ~10 if there are many entries.
+Launch verification subagents in one batch so they run concurrently. Group into batches of ~10 if there are many entries.
 
 **Step 2 — Collect results:** Read each agent's returned summary.
 
-**Step 3 — Apply edits sequentially:** Using the lookup results, apply one targeted edit at a time. Edits MUST be sequential (parallel edits to the same file cause conflicts).
+**Step 3 — Apply edits sequentially using `edit.py`:** Using the lookup results, build a patches.json and run `python3 $TOOLS_DIR/edit.py <file.bib> <patches.json>` one entry at a time. Do NOT edit the .bib file directly with agent editing tools such as Claude Code Edit or Codex `apply_patch`. Edits MUST be sequential (parallel edits to the same file cause conflicts).
 
 **Example agent prompt:**
 ```
@@ -160,13 +166,14 @@ For each `@article`, `@inproceedings`, `@book`, etc.:
 | Missing `% bibtidy:` URL | Every changed entry needs a URL — use DOI URL or venue page |
 | Incomplete commented original | Comment out ALL lines of the original, including closing `}` |
 | Adding comments to unchanged entries | Only changed entries get bibtidy comments — if compare.py reports zero mismatches and no error, do not touch the entry |
-| Rewriting entire file | Use one targeted edit per entry |
+| Rewriting entire file | Use `edit.py` with one entry or a small batch at a time; never rewrite the `.bib` file directly |
 | Deleting duplicate entries | Flag with comment only — never delete |
 | Losing `@string`/`@preamble` blocks | Preserve verbatim, don't touch |
 | Single hyphen in page ranges | Always use `--` (double hyphen) for BibTeX page ranges |
 | Partially applying title changes | When CrossRef title differs (e.g. preprint→published), replace the ENTIRE title with the CrossRef value — do not edit substrings |
 | Ignoring `number` field mismatches | `compare.py` reports `number` mismatches — apply them |
 | Adding `doi` when entry didn't have one | Never inject a `doi` field into an entry that lacks one |
+| Using agent editing tools instead of `edit.py` | Always use `edit.py` to apply changes, never edit the .bib file directly with Claude Code Edit, Codex `apply_patch`, or similar tools. Direct edits skip the commented original, URLs, and explanation |
 
 ## Preserve
 

tests/bibtidy/fixtures/expected.bib

@@ -153,6 +153,26 @@ @article{khader2022medical
   year={2023}
 }
 
+% Author list uses "and others" instead of full list
+% @inproceedings{kirillov2023segment,
+%   title={Segment anything},
+%   author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C and Lo, Wan-Yen and others},
+%   booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
+%   pages={3992--4003},
+%   year={2023},
+%   doi={10.1109/ICCV51070.2023.00371}
+% }
+% bibtidy: https://doi.org/10.1109/ICCV51070.2023.00371
+% bibtidy: expanded author list from "and others" to full list
+@inproceedings{kirillov2023segment,
+  title={Segment anything},
+  author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C and Lo, Wan-Yen and Doll{\'a}r, Piotr and Girshick, Ross},
+  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
+  pages={3992--4003},
+  year={2023},
+  doi={10.1109/ICCV51070.2023.00371}
+}
+
 % Published article should not be downgraded to preprint
 @article{tzou2022coronavirus,
   title={Coronavirus Resistance Database (CoV-RDB): SARS-CoV-2 susceptibility to monoclonal antibodies, convalescent plasma, and plasma from vaccinated persons},

tests/bibtidy/fixtures/input.bib

@@ -96,6 +96,16 @@ @article{khader2022medical
   year={2022}
 }
 
+% Author list uses "and others" instead of full list
+@inproceedings{kirillov2023segment,
+  title={Segment anything},
+  author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C and Lo, Wan-Yen and others},
+  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
+  pages={3992--4003},
+  year={2023},
+  doi={10.1109/ICCV51070.2023.00371}
+}
+
 % Published article should not be downgraded to preprint
 @article{tzou2022coronavirus,
   title={Coronavirus Resistance Database (CoV-RDB): SARS-CoV-2 susceptibility to monoclonal antibodies, convalescent plasma, and plasma from vaccinated persons},

tests/bibtidy/test_validate.py

@@ -164,6 +164,16 @@ def test_parenthesized_entry_rejected(self):
         "volume": (None, "13"),
         "year": ("2022", "2023"),
     },
+    "kirillov2023segment": {
+        "author": (
+            "Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi"
+            " and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer"
+            " and Berg, Alexander C and Lo, Wan-Yen and others",
+            "Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi"
+            " and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer"
+            " and Berg, Alexander C and Lo, Wan-Yen and Doll{\\'a}r, Piotr and Girshick, Ross",
+        ),
+    },
     "tzou2022coronavirus": {},
     "aichberger2025semantically": {
         "title": (

tests/bibtidy/validate.py

@@ -293,6 +293,21 @@ def check_hallucinated_metadata_fixed(text):
     return t
 
 
+def check_author_expansion(text):
+    """Author list with 'and others' should be expanded with commented original + source."""
+    t = TestResult("Author list expanded (kirillov2023segment)")
+    entry = find_entry_block(text, "kirillov2023segment")
+    t.check(entry is not None, "Entry still exists")
+    t.check(find_commented_entry(text, "kirillov2023segment"), "Original entry commented out")
+    t.check(has_url(text, "kirillov2023segment"), "URL provided")
+    if entry:
+        author = get_field(entry, "author") or ""
+        t.check("and others" not in author, "'and others' removed")
+        t.check("Kirillov" in author, "First author preserved")
+        t.check(author.count(" and ") > 10, "Full author list present (>10 'and' separators)")
+    return t
+
+
 def check_published_article_not_downgraded(text):
     """Published article should not be downgraded to a preprint."""
     t = TestResult("Published article not downgraded (tzou2022coronavirus)")
@@ -315,7 +330,7 @@ def test_entry_count(text):
     cleaned = remove_special_blocks(text)
     cleaned = re.sub(r"(?m)^[ \t]*%.*$", "", cleaned)
     all_at = re.findall(r"^[ \t]*@(\w+)\{", cleaned, re.MULTILINE)
-    t.check(len(all_at) == 10, f"Expected 10 active entries, found {len(all_at)}")
+    t.check(len(all_at) == 11, f"Expected 11 active entries, found {len(all_at)}")
     return t
 
 
@@ -350,6 +365,7 @@ def main():
         check_wrong_pages_fixed(text),
         check_title_change_upgraded(text),
         check_hallucinated_metadata_fixed(text),
+        check_author_expansion(text),
         check_published_article_not_downgraded(text),
         check_hallucinated_entry_flagged(text),
     ]