Auto-load workflow templates and tighten PR template by Codex · Pull Request #106 · groupthinking/self-correcting-executor

Codex · 2026-03-20T10:43:04Z

Title

Auto-load workflow templates and tighten PR template

Summary

What changed: Added data-driven workflow template loading (yaml/json under workflows/data/), seeded WRKFLW-derived construction log template, and refreshed PR template for concise summaries.
Why it helps: Makes the workflow catalog extensible without code changes and improves PR signal-to-noise for reviewers.

Orchestration example (optional):

engine.execute_intent(
    "construction daily log",
    options={"workflow_template": "construction_daily_log_external"},
)

Type

Checklist

Tests added/updated
Tests passing locally
Docs updated (if applicable)
Follows project style

Sections

Changes:
- Auto-load external workflow templates from workflows/data (yaml/json) into the catalog.
- Added WRKFLW PRD–based construction_daily_log_external template.
- Documented extension path in consolidation docs and README.
- PR template trimmed to concise summary, type/checklist, and optional orchestration snippet.
Testing:
- python -m pytest (new external-template coverage included).
Notes:
- Deprecation warnings about datetime.utcnow() remain unchanged.

Original prompt

This section details on the original issue you should resolve

<issue_title>🔴 Absorb WRKFLW and workflows repos — become the canonical workflow engine</issue_title>
<issue_description>## Problem
3 workflow execution repos:

self-correcting-executor (1.3MB, 15 issues) — THIS REPO, has auto-fix capability

WRKFLW (451KB, 14 issues) — TypeScript workflow engine

workflows (156KB, 1 issue) — TypeScript workflow definitions

Unique Value

self-correcting-executor's auto-fix capability is the differentiator — workflows that detect failures and self-heal. WRKFLW and workflows have generic workflow definitions that should be absorbed.

Plan

Port WRKFLW's workflow templates into self-correcting-executor

Port workflows' definitions as examples

Archive both source repos

Acceptance Criteria

All workflow patterns from 3 repos consolidated here

Self-correction engine wraps imported workflow types

WRKFLW and workflows archived</issue_description>

Comments on the Issue (you are @codex[agent] in this section)

Fixes 🔴 Absorb WRKFLW and workflows repos — become the canonical workflow engine #105

Copilot

Pull request overview

Consolidates workflow templates from external repos into a new workflows package and integrates template selection + self-healing retries into OrchestrationEngine, along with supporting docs and tests.

Changes:

Added a declarative workflow template catalog + registry helpers (workflows/) and wired template materialization into orchestration.
Implemented step-level retry/self-heal behavior and failure recording hooks during workflow execution.
Added documentation and tests validating template coverage and orchestrator template selection; updated test dependencies for async tests.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
workflows/templates.py	Introduces `WorkflowTemplate` and consolidated template definitions.
workflows/self_correcting.py	Adds helper to apply retry policy defaults to workflow steps.
workflows/registry.py	Provides template listing, intent-to-template matching, and materialization.
workflows/init.py	Exposes registry functions as package API.
orchestrator.py	Integrates template generation, applies self-correction, adds retry loop + failure recording.
tests/test_workflow_templates.py	Adds tests for template sources, materialization, and orchestrator selection.
tests/test_mcp_compliance.py	Adjusts MCP fixtures to be synchronous.
tests/conftest.py	Ensures repo root is on `sys.path` for test imports.
requirements-test.txt	Adds `pytest-asyncio` to test requirements.
mcp_server/real_mcp_server.py	Adds tool registry + lightweight `RealMCPServer` wrapper for tests.
docs/workflows/CONSOLIDATION.md	Documents consolidation and template-driven orchestration usage.
README.md	Mentions the new consolidated workflow catalog.
MCP_INTEGRATION_PLAN.md	Clarifies “no mocks” policy statement.
config/init.py	Adds config package marker.
.github/dependabot_test.py	Adds helper module used by tests for dependabot config assertions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-20T18:39:40Z

requirements-test.txt

@@ -1,4 +1,5 @@
 pytest>=6.0.0


pytest-asyncio>=0.23.0 is likely incompatible with pytest>=6.0.0, and it also diverges from requirements-ci.txt (which requires pytest>=7). To avoid resolver conflicts and keep local/CI aligned, bump the pytest lower bound here to match CI (or adjust pytest-asyncio version accordingly).

Suggested change

pytest>=6.0.0

pytest>=7.0.0

Copilot · 2026-03-20T18:39:40Z

workflows/self_correcting.py

+    policy = workflow.get("self_correction", {}).get("policy", {})
+    max_attempts = policy.get("max_attempts", 2)
+
+    for step in workflow.get("steps", []):
+        retry = step.get("retry_policy") or {}
+        retry.setdefault("max_attempts", max_attempts)
+        retry.setdefault("strategy", "self_heal")
+        step["retry_policy"] = retry


apply_self_correction ignores the self_correction.enabled flag that templates set, and it also applies a default retry policy even when self_correction is absent. This makes it impossible to opt out of retries and leaves enabled as dead metadata. Consider short-circuiting when self_correction.enabled is false (or missing), and only defaulting max_attempts when self-correction is enabled.

Copilot · 2026-03-20T18:39:41Z

workflows/templates.py

+
+        for step in self.steps:
+            step_copy = dict(step)
+            step_copy.setdefault("retry_policy", default_retry)


default_retry is a single dict reused across all steps via setdefault, so multiple steps will share the same retry_policy object. If any code later mutates one step’s retry policy (e.g., per-step backoff, attempts), it will unintentionally affect all steps. Create a new dict per step (copy/merge) when assigning the default retry policy.

Suggested change

step_copy.setdefault("retry_policy", default_retry)

if "retry_policy" not in step_copy:

step_copy["retry_policy"] = default_retry.copy()

Copilot · 2026-03-20T18:39:41Z

workflows/templates.py

+            "construction",
+            "daily log",
+            "rfi",
+            "wrklfw",


The tag wrklfw looks like a misspelling of wrkflw (repo name). Since match_template_for_intent does substring matches on tags, this typo can prevent intent matching based on that tag.

Suggested change

"wrklfw",

"wrkflw",

Copilot · 2026-03-20T18:39:41Z

orchestrator.py

    async def store_execution(self, data: Dict):
        """Store execution data in graph"""
        node_id = data["workflow_id"]
        self.graph[node_id] = {
            "data": data,
            "relationships": [],
            "insights": [],
+            "failures": [],
        }


store_execution overwrites any existing node for the same workflow_id (including failures recorded during execution), resetting failures to an empty list. This loses failure history that _handle_step_failure just recorded. Consider merging into an existing node (preserve failures/relationships/insights) instead of replacing it.

Copilot · 2026-03-20T18:39:41Z

orchestrator.py

+        node = self.graph.setdefault(
+            workflow_id,
+            {"data": {}, "relationships": [], "insights": [], "failures": []},
+        )


record_step_failure can create a graph node with data: {}. find_similar_executions later assumes node['data']['intent'] exists, which will raise a KeyError if called before store_execution populates data. Either ensure data includes the intent when creating the node, or guard against missing intent in find_similar_executions.

Copilot · 2026-03-20T18:39:42Z

docs/workflows/CONSOLIDATION.md

+```python
+from orchestrator import OrchestrationEngine
+
+engine = OrchestrationEngine()
+result = asyncio.run(
+    engine.execute_intent(
+        "Generate a construction daily log workflow",
+        options={"workflow_template": "construction_daily_log"},
+    )
+)


The example uses asyncio.run(...) but doesn’t import asyncio, so the snippet won’t run as written. Add the missing import (or adjust the snippet).

sentry · 2026-03-20T18:45:09Z

orchestrator.py

+            attempt = 0
+            last_error = None
+
+            while attempt < attempts:


Bug: If a step's retry_policy has max_attempts set to 0, the execution loop is never entered, causing the step to be silently skipped without any logging or errors.
_{Severity: HIGH}

Suggested Fix

Validate the max_attempts value when loading or materializing workflow templates. Ensure that max_attempts is a positive integer, defaulting to 1 if it is not set or is invalid. Reject or correct templates where max_attempts is 0.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: orchestrator.py#L289 Potential issue: The retry logic in the step executor uses a `while attempt < attempts:` loop. If a workflow step is configured with `"max_attempts": 0`, this condition is immediately false. Consequently, the step's execution block is never entered. This results in the step being silently skipped without any execution, logging, or error reporting. The workflow proceeds as if the step succeeded, eventually marking the entire workflow as "completed". This can lead to silent failures and incomplete data processing, particularly since the pull request allows loading external templates which may not have validated `max_attempts` values.

_{Did we get this right? 👍 / 👎 to inform future reviews.}

Initial plan

621b029

Codex AI assigned Codex and groupthinking Mar 20, 2026

Codex started work on behalf of groupthinking March 20, 2026 11:10 View session

feat: consolidate workflow templates

4ad9905

Codex AI changed the title ~~[WIP] Absorb WRKFLW and workflows repos into self-correcting-executor~~ Consolidate workflow templates with self-healing orchestration Mar 20, 2026

Codex AI requested a review from groupthinking March 20, 2026 11:23

Codex finished work on behalf of groupthinking March 20, 2026 11:23

groupthinking approved these changes Mar 20, 2026

View reviewed changes

groupthinking marked this pull request as ready for review March 20, 2026 18:35

Merge branch 'master' into codex/absorb-wrkflw-and-workflows

ae0f62b

Copilot AI review requested due to automatic review settings March 20, 2026 18:35

Copilot started reviewing on behalf of groupthinking March 20, 2026 18:36 View session

Codex started work on behalf of groupthinking March 20, 2026 18:36 View session

Copilot AI reviewed Mar 20, 2026

View reviewed changes

chore: load external workflow templates and update pr template

6c04552

Codex AI changed the title ~~Consolidate workflow templates with self-healing orchestration~~ Auto-load workflow templates and tighten PR template Mar 20, 2026

Codex AI requested a review from groupthinking March 20, 2026 18:42

Codex finished work on behalf of groupthinking March 20, 2026 18:42

sentry bot reviewed Mar 20, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Auto-load workflow templates and tighten PR template#106

Auto-load workflow templates and tighten PR template#106
Codex wants to merge 4 commits intomasterfrom
codex/absorb-wrkflw-and-workflows

Codex AI commented Mar 20, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 20, 2026

Uh oh!

Copilot AI Mar 20, 2026

Uh oh!

Copilot AI Mar 20, 2026

Uh oh!

Copilot AI Mar 20, 2026

Uh oh!

Copilot AI Mar 20, 2026

Uh oh!

Copilot AI Mar 20, 2026

Uh oh!

Copilot AI Mar 20, 2026

Uh oh!

sentry bot Mar 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	step_copy.setdefault("retry_policy", default_retry)
	if "retry_policy" not in step_copy:
	step_copy["retry_policy"] = default_retry.copy()

Conversation

Codex AI commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Title

Summary

Type

Checklist

Sections

Unique Value

Plan

Acceptance Criteria

Comments on the Issue (you are @codex[agent] in this section)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

sentry bot Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Codex AI commented Mar 20, 2026 •

edited

Loading