[prompt-analysis] Copilot PR Prompt Analysis - February 17, 2026

[github-actions[bot]]

Executive Summary

Analysis Period: Last 30 days (1,000 PRs analyzed)
Overall Performance: 698 merged (69.9% success rate) | 300 closed | 2 open

Key Finding: Bug fix prompts dominate the dataset (76.4% of all PRs) and show strong performance with a 71.1% success rate. However, documentation prompts, while rare (0.7%), achieve the highest success rate at 85.7%.

Key Metrics

Metric	Value
Total PRs Analyzed	1,000
Successfully Merged	698 (69.9%)
Closed Without Merge	300
Currently Open	2

Success Rates by Prompt Category

Category	Total PRs	Merged	Closed	Success Rate
Documentation	7	6	1	85.7%
Other	12	9	3	75.0%
Bug Fix	764	543	221	71.1%
Refactor	6	4	2	66.7%
Feature	180	118	62	65.6%
Update	14	9	5	64.3%
Testing	15	9	6	60.0%

Pattern Analysis

✅ Successful Prompt Characteristics

High Success Rate Categories:

1. Documentation (85.7% success) - Clear, specific documentation tasks
2. Other/Miscellaneous (75.0% success) - Unique, well-defined tasks
3. Bug Fix (71.1% success) - Problem-solving with clear objectives

Common Success Factors:

• Specificity: Successful prompts reference specific files, functions, or error messages
• Clarity: Clear problem statements and expected outcomes
• Action-oriented: Strong action verbs (fix, add, update, create)
• Context: Include relevant error messages, reproduction steps, or requirements

Most Common Keywords in Merged PRs:

• test: 2,490 occurrences
• add: 743 occurrences
• update: 641 occurrences
• create: 510 occurrences
• fix: 466 occurrences
• resolve: 268 occurrences
• remove: 172 occurrences
• implement: 104 occurrences
• correct: 103 occurrences
• bump: 84 occurrences

❌ Closed PR Patterns

Lower Success Categories:

1. Testing (60.0% success) - May require more specificity about test scope
2. Update (64.3% success) - Version bumps and dependency updates

Characteristics of Closed PRs:

• Longer prompts for features: Feature prompts that were closed averaged 276.9 words vs 256.6 for merged
• Less specific requests: Vague improvement requests without clear criteria
• Missing context: Lack of specific files, functions, or error details

View Example Successful Prompts

Bug Fix Examples (71.1% success rate)

Example 1: PR #3784

• Title: Fix panic in expression validation when step ID contains hyphens
• Prompt excerpt: Fix a panic that occurs during expression validation when a step ID contains hyphens.

The issue is in the extractStepReferences function in pkg/workflow/expression_extraction.go. Th...

Example 2: PR #3782

• Title: Fix error handling in workflow compilation when default branch detection fails
• Prompt excerpt: Fix an issue where the workflow compiler panics if the default branch cannot be determined (e.g., when not in a git repository or when git command fails).

Currently, NewCompiler ...

Feature Examples (65.6% success rate)

Example 1: PR #3779

• Title: Add support for custom step timeout configuration in workflow frontmatter
• Prompt excerpt: Add support for configuring custom timeouts for individual steps in workflow frontmatter.

Currently, all steps use a default timeout, which can be problematic for long-running operatio...

Example 2: PR #3773

• Title: Add YAML validation mode for gh-aw compile command
• Prompt excerpt: Add a --validate-only or --check flag to the gh aw compile command that:

1. Validates the YAML syntax and structure without writing the output file
2. Reports any validation er...

View Example Closed Prompts

Bug Fix Examples That Were Closed

Example 1: PR #3766

• Title: Fix MCP server initialization race condition in concurrent workflows
• Prompt excerpt: Fix a race condition that occurs when multiple workflows attempt to initialize the same MCP server concurrently.

Current Issue:
When multiple workflow runs try to start the s...

Example 2: PR #3756

• Title: Fix memory leak in workflow compilation cache
• Prompt excerpt: Fix a memory leak in the workflow compilation cache that occurs when compiling large workflows repeatedly.

Problem:
The Compiler struct maintains an internal cache of parsed wo...

Feature Examples That Were Closed

Example 1: PR #3768

• Title: Add support for conditional MCP server initialization based on runtime flags
• Prompt excerpt: Add support for conditional MCP server initialization in workflows, allowing users to enable/disable specific MCP servers based on runtime flags or environment variables.

Motivation:
Some...

Example 2: PR #3760

• Title: Add workflow dependency graph visualization to gh-aw CLI
• Prompt excerpt: Add a new gh aw graph command that generates a visual dependency graph of workflow steps and jobs.

Motivation:
For complex workflows with many custom jobs and step dependencies, i...

Key Insights

Based on the analysis of 1,000 Copilot-generated PRs:

1. Bug Fixes Dominate and Succeed: 764 PRs (76.4%) were bug fixes with a strong 71.1% success rate. The keyword "test" appears 2,490 times in merged PRs, suggesting successful bug fixes often include test updates.

2. Documentation Has Highest Success: Only 7 documentation PRs were created, but 85.7% were merged. This suggests clear, bounded documentation tasks are highly successful.

3. Feature Prompts Need More Specificity: Feature additions have a lower success rate (65.6%) and closed feature PRs had longer prompts (276.9 vs 256.6 words), suggesting verbosity without clarity doesn't help.

4. Action Verbs Matter: Top keywords in merged PRs are action-oriented: test (2,490), add (743), update (641), create (510), fix (466). These verbs appear consistently in successful prompts.

5. Test Coverage is Critical: The keyword "test" dominates both merged (2,490) and closed (893) PRs, but appears 2.8x more in merged PRs, indicating successful PRs often include comprehensive test coverage.

Recommendations

DO ✅:

1. Be Specific: Reference exact file paths, function names, or error messages

   • ✅ "Fix the authentication error in pkg/auth/validator.go line 42"
   • ❌ "Improve authentication handling"

2. Include Tests: Explicitly mention test requirements or updates

   • ✅ "Add validation and update tests in validator_test.go"
   • ❌ "Add validation logic"

3. Use Clear Action Verbs: Start with fix, add, update, create, implement

   • ✅ "Fix nil pointer dereference in user service"
   • ❌ "Handle user service issues better"

4. Provide Context: Include error messages, reproduction steps, or requirements

   • ✅ "Fix panic when username is empty (error: 'runtime error: invalid memory address')"
   • ❌ "Fix panic in auth"

AVOID ❌:

1. Vague Improvements: "Make it better" or "Improve performance" without metrics
2. Long Rambling Prompts: Feature requests over 300 words often get closed
3. Missing Scope: Requests without clear boundaries or completion criteria
4. No File References: Generic requests without specific code locations

Historical Trends

View 7-Day Trend Analysis

Date	Total PRs	Merged	Closed	Success Rate
2026-02-17	1000	698	300	69.9%
2026-02-16	1000	697	301	69.8%
2026-02-15	1000	695	303	69.6%
2026-02-14	1000	696	302	69.7%
2026-02-13	1000	695	303	69.6%
2026-02-12	1000	694	304	69.5%
2026-02-11	1000	693	305	69.4%

Trend: ↗️ Success rate increased by 0.1 percentage points compared to the previous analysis.

Prompt Templates

Based on successful patterns, here are recommended templates:

Bug Fix Template (71.1% success rate)

Fix [specific error/issue] in [file path]

**Problem**: [Error message or behavior]
**Location**: [File:line or function name]
**Expected**: [What should happen]

**Changes needed**:
- Fix [specific issue]
- Add/update tests in [test file]
- Update error handling if needed

Feature Addition Template (65.6% success rate)

Add [specific feature] to [component/file]

**Requirements**:
- [Specific requirement 1]
- [Specific requirement 2]

**Files to modify**:
- [file1.go]: Add [functionality]
- [file1_test.go]: Add test coverage

**Acceptance criteria**:
- [Testable outcome 1]
- [Testable outcome 2]

---

Analysis Period: Last 30 days (rolling window)
Next Update: Daily automated analysis

AI generated by Copilot PR Prompt Pattern Analysis

• expires on Feb 24, 2026, 5:12 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-02-24T05:12:40.328Z.

Closed by Workflow