diffract

An OCaml library and CLI tool for parsing source files using tree-sitter and pattern matching with concrete syntax.

Features

• Parse source files to S-expressions using tree-sitter grammars
• Pattern matching with concrete syntax and metavariables
• Semantic patch transforms (find-and-replace at the AST level)
• Support for TypeScript, Kotlin, and other languages (extensible)

Building

Prerequisites

• OCaml 5.2+
• opam
• tree-sitter library (libtree-sitter) - see below
• npm (to fetch grammar sources)

Installing tree-sitter

macOS:

brew install tree-sitter

Ubuntu/Debian:

sudo apt install libtree-sitter-dev

Arch Linux:

sudo pacman -S tree-sitter

Build Steps

# Install OCaml dependencies (add --with-test to include test/benchmark deps)
opam install . --deps-only --with-test

# Build grammar libraries (TypeScript, Kotlin)
cd grammars && ./build-grammars.sh && cd ..

# Build the project
dune build

# Run tests
dune test

# Format code (requires ocamlformat: opam install ocamlformat)
dune fmt

Formatting is managed via ocamlformat. Run dune fmt to reformat all OCaml and dune files before committing.

Note: dune fmt emits "Stray '@'" warnings for @@ delimiters appearing in doc comments. These are harmless and can be ignored.

Usage

# Parse and print parsed tree
diffract example.ts

# Parse with explicit language
diffract --language kotlin example.kt

# Match a pattern against a single file
diffract --match pattern.txt source.ts

# Scan a directory for pattern matches
diffract --match pattern.txt --include '*.ts' src/

# Scan with custom directory exclusions
diffract --match pattern.txt --include '*.ts' -e vendor -e dist src/

# Apply a semantic patch (preview diff)
diffract --apply --match patch.txt source.ts

# Apply a semantic patch in place
diffract --apply --in-place --match patch.txt source.ts

# Apply across a directory
diffract --apply --match patch.txt --include '*.ts' src/

# List available languages
diffract --list-languages

Transforms (Semantic Patches)

Patterns can include -/+ prefixed lines to describe code transformations. For example, to rename console.log to logger.info:

patch.txt:

@@
match: strict
metavar $MSG: single
@@
- console.log($MSG)
+ logger.info($MSG)

$ diffract --apply --match patch.txt source.ts
--- a/source.ts
+++ b/source.ts
@@ -1,3 +1,3 @@
 function greet(name: string) {
-    console.log(name);
+    logger.info(name);
 }

Lines prefixed with - are matched and removed; lines with + are inserted. Unprefixed (or space-prefixed) lines are context that appears in both match and replace. Metavariables carry values from the match side to the replace side.

See Transform documentation for partial-mode and field-mode transforms.

Directory Scanning

When the target is a directory, use --include to specify which files to scan:

Option	Description
--include GLOB / -i	Glob pattern for files (e.g., *.ts, *.py). Required for directories.
--exclude DIR / -e	Directory names to skip (repeatable). Defaults: node_modules, .git, _build, target, __pycache__, .hg, .svn

Supported glob patterns:

• *.ts - files ending with .ts
• prefix* - files starting with prefix
• *suffix - files ending with suffix

Example output:

src/api/auth.ts:15: console.log("login")
  $msg = "login"

src/utils/logger.ts:8: console.log("initialized")
  $msg = "initialized"

Found 2 match(es) in 2 file(s) (scanned 47 files)

Documentation

• Pattern format and library API
• Architecture and internals
• Testing and benchmarks

Match Architecture (Quick Overview)

The matching pipeline is split into focused modules:

• match_parse handles @@ preambles, metavars, ellipsis expansion, and spatch line classification.
• match_engine performs the structural matching (strict, field, partial) and sequence metavars (except in partial mode).
• match_search drives traversal, nested pattern contexts, indexing, and formatting.
• match_transform computes edits from match results and applies them to source text.
• match exposes the public API surface.

License

GPL-3.0-or-later