Skip to content

Commit e0ac145

Browse files
committed
docs: add AI comparison docs
PACT-4279
1 parent dfd1a7e commit e0ac145

File tree

1 file changed

+41
-1
lines changed

1 file changed

+41
-1
lines changed

website/docs/docs/ai/using.md

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -414,4 +414,44 @@ In AI, context is key. The more relevant information you can provide the more ta
414414

415415
### 3. Use Test Templates
416416

417-
Test Templates reduce the amount of refactoring required after generation. If you start to see a common generation pattern you wish to avoid, update the Test Template to help PactFlow AI generate improved output.
417+
Test Templates reduce the amount of refactoring required after generation. If you start to see a common generation pattern you wish to avoid, update the Test Template to help PactFlow AI generate improved output.
418+
419+
## Comparing PactFlow AI with Other AI Tools
420+
421+
PactFlow AI is purpose-built for contract testing workflows. While general-purpose AI tools like ChatGPT, GitHub Copilot, and IDE-native assistants (e.g., Windsurf) offer broad utility, they are not optimized for *specific* domains and use cases like contract testing with Pact.
422+
423+
The table below outlines key differences between these tools to help you understand how PactFlow AI can be incorporated into your development process.
424+
425+
### Feature Comparison
426+
427+
| Feature | **PactFlow AI** | **ChatGPT / Claude** | **GitHub Copilot** | **IDE-native AI (e.g., Windsurf)** |
428+
|--------|------------------|----------------------|--------------------|-------------------------------|
429+
| **Primary Use Case** | Contract test generation, review, and maintenance | General knowledge, Q&A, prototyping | Inline code suggestions | IDE-embedded assistant for development tasks |
430+
| **Breadth of Application** | Focused on Pact | Wide range of dev and non-dev tasks | General-purpose coding | Full-stack code, terminal, file editing |
431+
| **Strength in General Coding** | ❌ Not built for general-purpose coding | ✅ Excellent at most languages and use cases | ✅ Great for scaffolding and boilerplate | ✅ Helps with app logic, CLI, tests, refactors |
432+
| **Integration Surface** | CLI (now), IDE via MCP (coming), UI (planned) | Chat/web/app with integrations | Deep IDE integration (VS Code, JetBrains), Web/UI | Native to Cursor, Cline, etc. |
433+
| **Contract Testing Support** | ✅ Pact-specific knowledge, updated DSLs, test structure and matcher guidance | ❌ Generic or outdated knowledge | ❌ May suggest invalid or old Pact patterns | ❌ No contract testing awareness |
434+
| **Contract Testing Maintenance** | ✅ Review, update, and fix existing Pact tests | ❌ Prompt-based only | ❌ Suggests code, not test upkeep | ❌ Not designed for test maintenance |
435+
| **TDD Alignment** | ✅ Encourages scoped, purposeful test creation | ❌ May reinforce ad hoc development | ✅ Scaffolds tests, but not goal-oriented | ⚠️ Speed-optimized, not test-structured |
436+
| **Human-in-the-loop Design** |AI suggestions are reviewable and optional | ⚠️ User must prompt and validate output | ✅ Autocomplete with human review | ⚠️ High-speed, low-friction suggestions |
437+
| **Governance & Control** |RBAC-aware, CLI and platform-controlled | ⚠️ Black-box models, limited traceability | ⚠️ Limited observability | ⚠️ Tool-dependent privacy and trust model |
438+
439+
### When to Use Each Tool
440+
441+
| Use Case | Recommended Tool |
442+
|----------|------------------|
443+
| Creating and maintaining Pact tests | **PactFlow AI** |
444+
| Prototyping or researching APIs, code patterns | **ChatGPT / Claude** |
445+
| Writing app logic, scaffolding components or full-stack development in IDE | **GitHub Copilot**, **IDE-native AI tools** |
446+
447+
### Summary
448+
449+
PactFlow AI is not designed to replace general-purpose AI tools. Instead, it complements them by offering deep, current knowledge of contract testing—particularly Pact.
450+
451+
While other tools can help with coding in general, they do not provide guidance on:
452+
- Pact test quality
453+
- Over-specification and drift detection
454+
- DSL correctness and best practices
455+
- And other contract-testing specific capabilities
456+
457+
For teams working with Pact, PactFlow AI acts as a domain expert—reviewing, generating, and maintaining tests with precision and context.

0 commit comments

Comments
 (0)