Parser combinator DSL + macro multiple-evaluation fixes#44
Open
brightprogrammer wants to merge 11 commits into
Open
Parser combinator DSL + macro multiple-evaluation fixes#44brightprogrammer wants to merge 11 commits into
brightprogrammer wants to merge 11 commits into
Conversation
…ontext Add Include/Misra/ParserCombinator.h: a documented, debuggable, scannerless parser-combinator DSL. Parsers are plain `static inline` functions; StrIter and PcParserStatus stay inside the block frames (PcSeq/PcChoice), the arms (PcAlt/PcTryAlt and their Then variants), and the two atoms (PcSatisfyChar / PcSatisfyStr) -- a grammar rule never names either one. Thread a per-grammar `ParserCtx` (carrying the allocator parsers build AST nodes from, plus the growing AST root / symbol tables) through every parser as `ctx`, enabling immediate-mode parsing: build long-lived output as the input is consumed. Bin/ParseC.c: an arithmetic calculator as the running example and playground for the DSL, to be grown into a full C parser.
…calculator DSL (ParserCombinator.h): - PcParse is OVERLOAD'd by arity, symmetric with PcParser (no __VA_ARGS__) - recognizer family for no-output "validator" parsers: PcRecognizer / PcRecognize / PcRecognizeMany - PcOpt: optional PcSeq step - PcReject: context-sensitive rule failure that hides the consumed/status bits - PcRun: driver-side entry point so a caller never spells pc_parser_<Name> CalC (Bin/CalC.c, renamed from ParseC.c): - immediate-mode calculator: Num (tagged i64/f64) with promotion, variables in a Map(Str, Num) on ParserCtx, unary minus, %, parens, scannerless whitespace, seeded pi/e - REPL or -c <expr> via ArgParse; success/error reported by the driver - tokens accumulate through the atoms into a Str and convert via StrToU64 / StrToF64 -- no forged Str views, and no grammar rule touches StrIter/status
… a tool Error reporting as a shared, greedy model (the rustc-style sink, not first-fail): DSL (ParserCombinator.h): - PcReport (level + input span + message) and PcReportError/Warn/Info: record a diagnostic to ctx->reports and keep going, so a rule poisons its value instead of unwinding and one input surfaces as many errors as it has. - macro-hygiene fixes surfaced by a fan-out audit: PcSatisfyStr now evaluates its Expect argument once (bound to a UNPL local); PcOpt is a single for-scoped statement (was an enclosing-scope decl + bare if), so it nests, sits one-per-line, and takes an unbraced body without a dangling-else. CalC (Bin/CalC.c): - greedy diagnostics: a poison bit on Num that propagates through arithmetic, undefined-variable / division-by-zero / overflow reported-and-continued, and a rustc-style renderer (source line + carets) that parsers never touch. - REPL prints results/errors via WriteFmt/WriteFmtLn (+ a prompt), not LOG. - the driver checks leftover input through the Iter API (StrIterRemainingLength / StrIterIndex), not raw field access. - assignment upserts (Map is a multimap, so replace not duplicate) and does not bind a poisoned result. Build: - vendor calc as a real Bin tool in meson.build (gated on Map + File, install:false), built through the existing build/ dir.
VecInsertL / VecInsertFastL (and everything routing through them -- VecPushBackL, StrPushBackL, ...) and ListInsertL passed the inserted value TWICE: `&LVAL_AS(T, lval)` (a stable copy to insert) and `&(lval)` (the source to zero on the move). A side-effecting lvalue like `VecPushBack(&v, arr[i++])` therefore ran its side effect twice. Collapse each to a single `&(lval)`: `vec_insert_one_l` / `list_insert_one_l` now take one `void *source` and do both the copy-in and the zero-on-success through it (the shape Map's L-form already used). LVAL_AS stays in the R-forms, which is where immediates/rvalues legitimately need it -- so `VecPushBack(v, <immediate>)` still (correctly) will not compile a move of a temporary. Full suite green (119/119).
- PcReportErrorHere/WarnHere/InfoHere: report a diagnostic spanning the single
character at the cursor (frame-free), for "expected X" errors where nothing was
consumed -- the twin of the frame-span PcReport* family.
- PcElse (PcChoice fallback: body runs iff no arm matched, without a rule naming
pc_ch) + PcRecover (resync skip); CalC's Atom uses them so a bad operand is
reported and the parse resyncs, surfacing many errors per line, not just the first.
- rename the diagnostic API to the module prefix: ReportLevel -> PcReportLevel,
REPORT_{INFO,WARN,ERROR} -> PC_REPORT_{INFO,WARN,ERROR}.
- PC_REPORT / PC_REPORT_HERE push through VecPushBackR (R-form) so the report's
compound literal, including its message, is evaluated once.
The reverse VecForeach*ReverseIdx macros wrapped the iteration loop in two run-once for-loops (to inject the var/run_once declarations), so a break in the body exited those wrappers, not the iteration loop -- iteration silently continued to the next element while forward variants stopped. Mirror the forward variants (iteration loop innermost) so break exits it; the shrink-during-iteration guard is preserved. Adds forward + reverse early-break tests.
…king rollback Weaves a savepoint contract into the DSL: a grammar defines PcParserCtx + PcParserCtxMark + PcParserCtxSnapshot/PcParserCtxRollback, and every combinator that can abandon an attempt (PcChoice arms, PcMatchZeroOrMore/PcMatchOneOrMore, PcOpt, the recognizer repeats) snapshots the context on entry and rolls it back when the attempt is abandoned -- a failed parse leaves the context untouched, a committed one keeps its mutations. The grammar author writes only mutations, never snapshot/rollback. Also renames the repetition combinators for honesty (PcMatchZeroOrMore/PcMatchOneOrMore, PcRecognizeZeroOrMore/PcRecognizeOneOrMore, adding the one-or-more variants) and ParserCtx -> PcParserCtx. CalC drops its Map for an append-only Vec(Binding) (rollback = truncate), and VarRef finds the newest binding with VecForeachPtrReverse + break.
…irst grammar
Deliver fixed-width byte-reader parsers out of the box so a binary grammar
composes them directly instead of hand-writing a reader per field: PcU8/PcI8
and Pc{U,I}{16,32,64}{BE,LE}, declared in the header and defined in
Source/Misra/ParserCombinator.c over a BufIter. They live outside the grammar's
translation unit, so a grammar's context must be a tagged `struct PcParserCtx`
for the opaque pointer to line up.
Add the combinators the arrays need: PcMatchExactlyN (index-bearing, as
VecForeachIdx is to VecForeach) and PcRecognizeExactlyN, plus PcExpect (the
recognizer twin of PcMatch) and PcSkipBytes.
Buf gains signed fixed-width reads (BufReadI<N>, two's-complement wrappers over
the unsigned readers) and parameter docs on every fixed-width reader.
Reimplement the Tzif resolver on the DSL: Header is a parser producing a value,
the version and transition-time width are PcChoice decisions rather than runtime
fields, each resolution phase is its own parser, and the context holds only the
instant being resolved. Validated by the existing Tzif test suite.
Removes the guidance requiring a `// intentional bypass:` comment on direct-field-write sites in corruption/Deadend tests, in both the "Accessor macros are read-only" and Deadend-fixtures sections.
ProcMaps now owns each entry's path as a NUL-terminated Str and drops the retained raw buffer after parsing -- no borrowed, non-terminated slices to reason about downstream. This dissolves the whole borrowed-slice hazard at its root: every consumer reads a real C-string. - File/Elf: add the (Zstr,len) fixed-length form as a fourth call shape alongside the existing Str* / Zstr / char* _Generic arms (char* is the MSVC string-literal synonym for Zstr -- see Zstr.h, kept intentionally). FileOpen, FileReadAndClose and ElfOpen gain a (Zstr,len) arity backed by file_open_n / file_read_and_close_*_n / elf_open_n, which copy the fixed-length view into a stack buffer only because open() needs a NUL-terminated C-string with no way to pass a bound. - SymbolResolver: cache owns its path Str copy; module_path borrows it. - Supporting DSL: PcCaptureUntil, PcFailIfNotEof, PcReportsRender, PcI<N>. - Tests + CalC updated for the owned path and rendered diagnostics.
Extend the string-input dispatch macros to the full four-branch StrCmp shape -- Str* / Zstr / char* (the MSVC literal synonym, kept) plus a (Zstr, len) fixed-length arity -- across: Dir, Dns, KvConfig, MachO, Http, Pe, Pdb, Socket. Each (Zstr, len) form routes to a `_cstr` worker that makes a transient copy ONLY when required: - byte compare/lookup (MachO/Pe/Http section+header keys) threads the bound via ZstrCompareN with an exact-length guard -- no copy; - Str-keyed lookups (KvConfig) use the existing StrInitFromCstr idiom (mirroring the _zstr sibling), not a hand-built Str view; - NUL-terminating syscalls (Dir open/mkdir/unlink, MachO/Pe/Pdb open, Socket connect, Http file) copy into a bounded stack buffer. Also fixes a latent prefix false-match in macho_find_section_cstr and keeps private snake_case workers in each module's Private.h.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ongoing branch for the parser-combinator DSL.
What's here now:
playground.
(PcParserCtx + PcParserCtxMark + PcParserCtxSnapshot/PcParserCtxRollback), and
every combinator that can abandon an attempt (PcChoice arms, the repetitions,
PcOpt, the recognizer repeats) snapshots the context on entry and rolls it
back when the attempt is abandoned -- a failed parse never leaks its
mutations, a committed one keeps them. The grammar author writes only
mutations, never snapshot/rollback.
PcMatchOneOrMore and PcRecognizeZeroOrMore / PcRecognizeOneOrMore (added the
one-or-more forms). ParserCtx -> PcParserCtx.
truncate) -- the rollback-friendly shape the savepoint model needs.
breaknow exits reverse VecForeachloops (they wrapped iteration in run-once loops and swallowed break). Same fix
for the other containers tracked in [Bug] break does not exit reverse Foreach loops in List/Map/Str/Graph/BitVec #45.
Strcopyof its path (the raw buffer is dropped after parsing) — no borrowed,
non-terminated slices downstream.
StrCmpshape(
Str*/Zstr/char*+ a(Zstr, len)arity) for File/Elf/ProcMaps andthe Sys/Parser modules (Dir, Dns, KvConfig, MachO, Http, Pe, Pdb, Socket).
char*is kept as the MSVC literal synonym. Remaining stdlib containers aretracked in Sweep: four-branch (Str*/Zstr/char*/(Zstr,len)) dispatch for remaining string-input APIs #46.
Issues
breakin reverseVecForeachfor the other containersWIP — do not merge.