Skip to content

Commit 3cabd50

Browse files
Andrew-SchetininAndrew-Schetinin
authored
feat(#16): populate test data, validation oracle, and docker test script (#17)
1 parent e83b80e commit 3cabd50

21 files changed

Lines changed: 2342 additions & 7 deletions

File tree

‎.claude/commands/opsx/sync.md‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
---
2+
name: "OPSX: Sync"
3+
description: Sync delta specs from a change to main specs
4+
category: Workflow
5+
tags: [workflow, specs, experimental]
6+
---
7+
8+
Sync delta specs from a change to main specs.
9+
10+
This is an **agent-driven** operation - you will read delta specs and directly edit main specs to apply the changes. This allows intelligent merging (e.g., adding a scenario without copying the entire requirement).
11+
12+
**Input**: Optionally specify a change name after `/opsx:sync` (e.g., `/opsx:sync add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
13+
14+
**Steps**
15+
16+
1. **If no change name provided, prompt for selection**
17+
18+
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
19+
20+
Show changes that have delta specs (under `specs/` directory).
21+
22+
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
23+
24+
2. **Find delta specs**
25+
26+
Look for delta spec files in `openspec/changes/<name>/specs/*/spec.md`.
27+
28+
Each delta spec file contains sections like:
29+
- `## ADDED Requirements` - New requirements to add
30+
- `## MODIFIED Requirements` - Changes to existing requirements
31+
- `## REMOVED Requirements` - Requirements to remove
32+
- `## RENAMED Requirements` - Requirements to rename (FROM:/TO: format)
33+
34+
If no delta specs found, inform user and stop.
35+
36+
3. **For each delta spec, apply changes to main specs**
37+
38+
For each capability with a delta spec at `openspec/changes/<name>/specs/<capability>/spec.md`:
39+
40+
a. **Read the delta spec** to understand the intended changes
41+
42+
b. **Read the main spec** at `openspec/specs/<capability>/spec.md` (may not exist yet)
43+
44+
c. **Apply changes intelligently**:
45+
46+
**ADDED Requirements:**
47+
- If requirement doesn't exist in main spec → add it
48+
- If requirement already exists → update it to match (treat as implicit MODIFIED)
49+
50+
**MODIFIED Requirements:**
51+
- Find the requirement in main spec
52+
- Apply the changes - this can be:
53+
- Adding new scenarios (don't need to copy existing ones)
54+
- Modifying existing scenarios
55+
- Changing the requirement description
56+
- Preserve scenarios/content not mentioned in the delta
57+
58+
**REMOVED Requirements:**
59+
- Remove the entire requirement block from main spec
60+
61+
**RENAMED Requirements:**
62+
- Find the FROM requirement, rename to TO
63+
64+
d. **Create new main spec** if capability doesn't exist yet:
65+
- Create `openspec/specs/<capability>/spec.md`
66+
- Add Purpose section (can be brief, mark as TBD)
67+
- Add Requirements section with the ADDED requirements
68+
69+
4. **Show summary**
70+
71+
After applying all changes, summarize:
72+
- Which capabilities were updated
73+
- What changes were made (requirements added/modified/removed/renamed)
74+
75+
**Delta Spec Format Reference**
76+
77+
```markdown
78+
## ADDED Requirements
79+
80+
### Requirement: New Feature
81+
The system SHALL do something new.
82+
83+
#### Scenario: Basic case
84+
- **WHEN** user does X
85+
- **THEN** system does Y
86+
87+
## MODIFIED Requirements
88+
89+
### Requirement: Existing Feature
90+
#### Scenario: New scenario to add
91+
- **WHEN** user does A
92+
- **THEN** system does B
93+
94+
## REMOVED Requirements
95+
96+
### Requirement: Deprecated Feature
97+
98+
## RENAMED Requirements
99+
100+
- FROM: `### Requirement: Old Name`
101+
- TO: `### Requirement: New Name`
102+
```
103+
104+
**Key Principle: Intelligent Merging**
105+
106+
Unlike programmatic merging, you can apply **partial updates**:
107+
- To add a scenario, just include that scenario under MODIFIED - don't copy existing scenarios
108+
- The delta represents *intent*, not a wholesale replacement
109+
- Use your judgment to merge changes sensibly
110+
111+
**Output On Success**
112+
113+
```
114+
## Specs Synced: <change-name>
115+
116+
Updated main specs:
117+
118+
**<capability-1>**:
119+
- Added requirement: "New Feature"
120+
- Modified requirement: "Existing Feature" (added 1 scenario)
121+
122+
**<capability-2>**:
123+
- Created new spec file
124+
- Added requirement: "Another Feature"
125+
126+
Main specs are now updated. The change remains active - archive when implementation is complete.
127+
```
128+
129+
**Guardrails**
130+
- Read both delta and main specs before making changes
131+
- Preserve existing content not mentioned in delta
132+
- If something is unclear, ask for clarification
133+
- Show what you're changing as you go
134+
- The operation should be idempotent - running twice should give same result

‎.claude/commands/opsx/verify.md‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
---
2+
name: "OPSX: Verify"
3+
description: Verify implementation matches change artifacts before archiving
4+
category: Workflow
5+
tags: [workflow, verify, experimental]
6+
---
7+
8+
Verify that an implementation matches the change artifacts (specs, tasks, design).
9+
10+
**Input**: Optionally specify a change name after `/opsx:verify` (e.g., `/opsx:verify add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
11+
12+
**Steps**
13+
14+
1. **If no change name provided, prompt for selection**
15+
16+
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
17+
18+
Show changes that have implementation tasks (tasks artifact exists).
19+
Include the schema used for each change if available.
20+
Mark changes with incomplete tasks as "(In Progress)".
21+
22+
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
23+
24+
2. **Check status to understand the schema**
25+
```bash
26+
openspec status --change "<name>" --json
27+
```
28+
Parse the JSON to understand:
29+
- `schemaName`: The workflow being used (e.g., "spec-driven")
30+
- Which artifacts exist for this change
31+
32+
3. **Get the change directory and load artifacts**
33+
34+
```bash
35+
openspec instructions apply --change "<name>" --json
36+
```
37+
38+
This returns the change directory and `contextFiles` (artifact ID -> array of concrete file paths). Read all available artifacts from `contextFiles`.
39+
40+
4. **Initialize verification report structure**
41+
42+
Create a report structure with three dimensions:
43+
- **Completeness**: Track tasks and spec coverage
44+
- **Correctness**: Track requirement implementation and scenario coverage
45+
- **Coherence**: Track design adherence and pattern consistency
46+
47+
Each dimension can have CRITICAL, WARNING, or SUGGESTION issues.
48+
49+
5. **Verify Completeness**
50+
51+
**Task Completion**:
52+
- If `contextFiles.tasks` exists, read every file path in it
53+
- Parse checkboxes: `- [ ]` (incomplete) vs `- [x]` (complete)
54+
- Count complete vs total tasks
55+
- If incomplete tasks exist:
56+
- Add CRITICAL issue for each incomplete task
57+
- Recommendation: "Complete task: <description>" or "Mark as done if already implemented"
58+
59+
**Spec Coverage**:
60+
- If delta specs exist in `openspec/changes/<name>/specs/`:
61+
- Extract all requirements (marked with "### Requirement:")
62+
- For each requirement:
63+
- Search codebase for keywords related to the requirement
64+
- Assess if implementation likely exists
65+
- If requirements appear unimplemented:
66+
- Add CRITICAL issue: "Requirement not found: <requirement name>"
67+
- Recommendation: "Implement requirement X: <description>"
68+
69+
6. **Verify Correctness**
70+
71+
**Requirement Implementation Mapping**:
72+
- For each requirement from delta specs:
73+
- Search codebase for implementation evidence
74+
- If found, note file paths and line ranges
75+
- Assess if implementation matches requirement intent
76+
- If divergence detected:
77+
- Add WARNING: "Implementation may diverge from spec: <details>"
78+
- Recommendation: "Review <file>:<lines> against requirement X"
79+
80+
**Scenario Coverage**:
81+
- For each scenario in delta specs (marked with "#### Scenario:"):
82+
- Check if conditions are handled in code
83+
- Check if tests exist covering the scenario
84+
- If scenario appears uncovered:
85+
- Add WARNING: "Scenario not covered: <scenario name>"
86+
- Recommendation: "Add test or implementation for scenario: <description>"
87+
88+
7. **Verify Coherence**
89+
90+
**Design Adherence**:
91+
- If `contextFiles.design` exists:
92+
- Extract key decisions (look for sections like "Decision:", "Approach:", "Architecture:")
93+
- Verify implementation follows those decisions
94+
- If contradiction detected:
95+
- Add WARNING: "Design decision not followed: <decision>"
96+
- Recommendation: "Update implementation or revise design.md to match reality"
97+
- If no design.md: Skip design adherence check, note "No design.md to verify against"
98+
99+
**Code Pattern Consistency**:
100+
- Review new code for consistency with project patterns
101+
- Check file naming, directory structure, coding style
102+
- If significant deviations found:
103+
- Add SUGGESTION: "Code pattern deviation: <details>"
104+
- Recommendation: "Consider following project pattern: <example>"
105+
106+
8. **Generate Verification Report**
107+
108+
**Summary Scorecard**:
109+
```
110+
## Verification Report: <change-name>
111+
112+
### Summary
113+
| Dimension | Status |
114+
|--------------|------------------|
115+
| Completeness | X/Y tasks, N reqs|
116+
| Correctness | M/N reqs covered |
117+
| Coherence | Followed/Issues |
118+
```
119+
120+
**Issues by Priority**:
121+
122+
1. **CRITICAL** (Must fix before archive):
123+
- Incomplete tasks
124+
- Missing requirement implementations
125+
- Each with specific, actionable recommendation
126+
127+
2. **WARNING** (Should fix):
128+
- Spec/design divergences
129+
- Missing scenario coverage
130+
- Each with specific recommendation
131+
132+
3. **SUGGESTION** (Nice to fix):
133+
- Pattern inconsistencies
134+
- Minor improvements
135+
- Each with specific recommendation
136+
137+
**Final Assessment**:
138+
- If CRITICAL issues: "X critical issue(s) found. Fix before archiving."
139+
- If only warnings: "No critical issues. Y warning(s) to consider. Ready for archive (with noted improvements)."
140+
- If all clear: "All checks passed. Ready for archive."
141+
142+
**Verification Heuristics**
143+
144+
- **Completeness**: Focus on objective checklist items (checkboxes, requirements list)
145+
- **Correctness**: Use keyword search, file path analysis, reasonable inference - don't require perfect certainty
146+
- **Coherence**: Look for glaring inconsistencies, don't nitpick style
147+
- **False Positives**: When uncertain, prefer SUGGESTION over WARNING, WARNING over CRITICAL
148+
- **Actionability**: Every issue must have a specific recommendation with file/line references where applicable
149+
150+
**Graceful Degradation**
151+
152+
- If only tasks.md exists: verify task completion only, skip spec/design checks
153+
- If tasks + specs exist: verify completeness and correctness, skip design
154+
- If full artifacts: verify all three dimensions
155+
- Always note which checks were skipped and why
156+
157+
**Output Format**
158+
159+
Use clear markdown with:
160+
- Table for summary scorecard
161+
- Grouped lists for issues (CRITICAL/WARNING/SUGGESTION)
162+
- Code references in format: `file.ts:123`
163+
- Specific, actionable recommendations
164+
- No vague suggestions like "consider reviewing"

0 commit comments

Comments
 (0)