Global learnings 2 #283
Closed
Global learnings 2 #283
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implements a global learnings aggregator that collects insights across all
iterations and islands, injecting them into prompts to help the LLM avoid
repeated mistakes and learn from successful patterns.
Features:
- Tracks common failures: syntax errors, runtime errors, performance regressions
- Optionally tracks successful patterns with improvement metrics
- Configurable aggregation window (default: last 50 iterations)
- Flexible injection (system/user prompts) with verbosity levels
- Full checkpoint save/load support
- Zero overhead when disabled (default: off)
Implementation Details:
- New GlobalLearningsConfig in config.py with 16+ configuration options
- GlobalLearnings class extracts patterns using regex on stderr/artifacts
- Pattern detection: NameError, IndexError, TypeError, SyntaxError, etc.
- Performance regression tracking (>10% metric decreases)
- Prompt integration via PromptSampler.build_prompt()
- ProcessParallelController passes learnings to workers and updates from results
- Checkpoint persistence to global_learnings.json
Configuration Example:
global_learnings:
enabled: true
track_failures: true
track_successes: false
window_size: 50
max_learnings: 5
inject_in_system_prompt: true
verbosity: "concise"
Output Format:
## Evolution Insights (Global Learnings)
### Common Pitfalls:
❌ Undefined variable: temp (seen 5x)
❌ IndexError: list index out of range (seen 3x)
⚠️ score decreased by 15.2% (seen 4x)
Bug Fixes:
- Fix worker process API key propagation by calling update_model_params()
with overwrite=True after LLMConfig reconstruction
Files Changed:
- openevolve/config.py: Add GlobalLearningsConfig
- openevolve/global_learnings.py: New core implementation
- openevolve/controller.py: Initialize and checkpoint management
- openevolve/prompt/sampler.py: Inject learnings into prompts
- openevolve/process_parallel.py: Worker integration and updates
- examples/config_with_global_learnings.yaml: Example config
- GLOBAL_LEARNINGS_IMPLEMENTATION.md: Complete documentation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
|
|
Author
|
@claude can you review this? |
Author
|
sorry, accidentally opened this on the main repo, meant to just do this on my forked version for now |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.