Skip to content

Commit dfdedf6

Browse files
feat(ai): PR 3 — Parsers + Execute Agent (#409)
Add agent-parser, extraction-parser, aggregation, and execute-agent modules with full unit test coverage. - agent-parser: parseStringResult, parseOpenCodeNDJSON, unwrapEnvelope (new shared export), unwrapAgentResult. Shared unwrapEnvelope breaks duplication between agent-parser and execute-agent (WIP fix #9). - extraction-parser: parseExtractionResult with multi-strategy JSON parsing (direct, markdown fence, pre-parsed object), and resolveImportPaths for prompt file resolution. - aggregation: normalizeJudgment, calculateRequiredPasses, aggregatePerAssertionResults with Zod validation. - execute-agent: extracted from ai-runner.js to break the circular dependency (ai-runner ↔ test-extractor). Logger injected at executeAgent call site rather than created inside spawnProcess (WIP fix #8). Uses shared unwrapEnvelope from agent-parser. - Test files use test.each for all table-driven cases per convention. 164 tests pass, 0 lint errors, TypeScript checks pass. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ai): address PR 3 code review findings - aggregation.js: validate once in aggregatePerAssertionResults — capture the Zod-validated result and compute Math.ceil inline, eliminating the redundant second schema parse inside calculateRequiredPasses - aggregation.js: remove misleading optional chaining (raw?.passed etc.) after the null-guard throw; use plain property access - agent-parser.js: replace acc.push() with [...acc, text] in reduce accumulator to prefer immutability per JS style guide - agent-parser.test.js: drop redundant "parsed object:" prefix from unwrapEnvelope test.each given fields; remove duplicate standalone "no result key" test that overlapped with test.each row - aggregation.test.js: remove redundant export-existence assertion for normalizeJudgment; add empty perAssertionResults edge case (vacuous truth — every() on [] returns true) - execute-agent.test.js: strengthen parseOutput test to verify stdout and logger are threaded through as expected (documents WIP fix #8) Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ai): address PR 3 author review findings - aggregation.js: rename `raw` param to `judgeResponse` and fold into single options object for normalizeJudgment; removes the two-argument signature (breaking change, callers updated) - aggregation.js: remove calculateRequiredPasses — math is inlined in aggregatePerAssertionResults, eliminating double schema parse - aggregation.test.js: remove calculateRequiredPasses describe block; fix Try() usage (direct fn ref, not arrow wrapper); update all normalizeJudgment call sites to new single-options signature - execute-agent.js: extract magic number 500 to maxOutputPreviewLength constant (camelCase per javascript.mdc); applied to all 3 truncation sites - execute-agent.test.js: replace try/catch antipatterns with await Try(); add Try import from riteway.js - extraction-parser.test.js: strengthen weak typeof assertions to check specific fields; strengthen cause !== undefined to cause.name === SyntaxError 151 tests pass, 0 lint errors, TypeScript clean. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ai): address PR 3 follow-up review findings - constants.js: rename calculateRequiredPassesSchema to aggregationParamsSchema — name now reflects what the schema validates (aggregation input params) rather than the deleted calculateRequiredPasses function; update all import sites - aggregation.test.js: add 6 missing Zod validation edge cases for aggregatePerAssertionResults (zero runs, negative runs, non-integer runs, NaN runs, negative threshold, NaN threshold) — coverage gap introduced when calculateRequiredPasses and its tests were removed; all cases now exercised via aggregatePerAssertionResults test.each 157 tests pass, 0 lint errors, TypeScript clean. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(test): complete PR review remediation 🐛 - Remove weak instanceof Error assertions 🔄 - Add threshold calculation verification tests Tests now verify threshold-based pass/fail logic directly 164 tests passing, 0 lint errors, TypeScript clean Co-authored-by: Ian White <ian.white.developer@gmail.com> * fix(ai): remove implementation detail from test - execute-agent.test.js: remove logger type assertion from parseOutput test — typeof checks violate tdd.mdc:64 and logger threading is an implementation detail; the three remaining assertions (call count, stdout arg, parsed result) collectively verify correct integration 164 tests pass, 0 lint errors, TypeScript clean. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 5bcb456 commit dfdedf6

10 files changed

Lines changed: 1843 additions & 5 deletions

source/agent-parser.js

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
import { createError } from 'error-causes';
2+
import { ParseError } from './ai-errors.js';
3+
4+
/**
5+
* Parse a string result from an agent, attempting multiple strategies:
6+
* 1. Direct JSON parse if string starts with { or [
7+
* 2. Extract and parse markdown-wrapped JSON (```json\n...\n```)
8+
* 3. Keep as plain text if neither works
9+
*/
10+
export const parseStringResult = (result, logger) => {
11+
const trimmed = result.trim();
12+
13+
if (trimmed.startsWith('{') || trimmed.startsWith('[')) {
14+
try {
15+
const parsed = JSON.parse(trimmed);
16+
logger.log('Successfully parsed string as JSON');
17+
return parsed;
18+
} catch {
19+
logger.log('Direct JSON parse failed, trying markdown extraction');
20+
}
21+
}
22+
23+
const markdownMatch = result.match(/```(?:json)?\s*\n([\s\S]*?)\n```/);
24+
if (markdownMatch) {
25+
logger.log('Found markdown-wrapped JSON, extracting...');
26+
try {
27+
const parsed = JSON.parse(markdownMatch[1]);
28+
logger.log('Successfully parsed markdown-wrapped JSON');
29+
return parsed;
30+
} catch {
31+
logger.log('Failed to parse markdown content, keeping original string');
32+
}
33+
}
34+
35+
logger.log('String is not valid JSON, keeping as plain text');
36+
return result;
37+
};
38+
39+
/**
40+
* Parse OpenCode's NDJSON output, extracting and concatenating all "text" events.
41+
*/
42+
export const parseOpenCodeNDJSON = (ndjson, logger) => {
43+
logger.log('Parsing OpenCode NDJSON output...');
44+
45+
const lines = ndjson.trim().split('\n').filter(line => line.trim());
46+
47+
const textEvents = lines.reduce((acc, line) => {
48+
try {
49+
const event = JSON.parse(line);
50+
if (event.type === 'text' && event.part?.text) {
51+
logger.log(`Found text event with ${event.part.text.length} characters`);
52+
return [...acc, event.part.text];
53+
}
54+
} catch (err) {
55+
logger.log(`Warning: Failed to parse NDJSON line: ${err.message}`);
56+
}
57+
return acc;
58+
}, []);
59+
60+
if (textEvents.length === 0) {
61+
throw createError({
62+
...ParseError,
63+
message: 'No text events found in OpenCode output',
64+
code: 'NO_TEXT_EVENTS',
65+
ndjsonLength: ndjson.length,
66+
linesProcessed: lines.length
67+
});
68+
}
69+
70+
const combinedText = textEvents.join('');
71+
logger.log(`Combined ${textEvents.length} text event(s) into ${combinedText.length} characters`);
72+
return combinedText;
73+
};
74+
75+
/**
76+
* Unwrap a JSON envelope object { result: ... }, returning the inner value.
77+
* If no envelope is present, returns the object as-is.
78+
* Shared helper used by unwrapAgentResult and execute-agent's raw output handling.
79+
*/
80+
export const unwrapEnvelope = (parsed) =>
81+
parsed?.result !== undefined ? parsed.result : parsed;
82+
83+
/**
84+
* Unwrap agent result from potential JSON envelope and parse nested JSON.
85+
* Handles Claude CLI's envelope format { result: "..." } and nested JSON strings.
86+
* @throws {Error} If output is not valid JSON after all parsing attempts
87+
*/
88+
export const unwrapAgentResult = (processedOutput, logger) => {
89+
const parsed = parseStringResult(processedOutput, logger);
90+
91+
if (typeof parsed === 'string') {
92+
throw createError({
93+
...ParseError,
94+
message: `Agent output is not valid JSON: ${parsed.slice(0, 100)}`,
95+
outputPreview: parsed.slice(0, 100)
96+
});
97+
}
98+
99+
const unwrapped = unwrapEnvelope(parsed);
100+
101+
logger.log(`Parsed result type: ${typeof unwrapped}`);
102+
if (typeof unwrapped === 'string') {
103+
logger.log('Result is string, attempting to parse as JSON');
104+
return parseStringResult(unwrapped, logger);
105+
}
106+
107+
return unwrapped;
108+
};

0 commit comments

Comments
 (0)