Support Hydra/OmegaConf composition in load_config

[thomashirtz]

Summary

It would be great if OpenEvolve supported Hydra/OmegaConf composition natively, so users can manage hierarchical configs, presets, and runtime overrides without hand-rolling YAML merges.

Motivation

• Current config path mixes plain YAML with dataclass loaders; selector-style files (with defaults) aren’t composed.
• Hydra enables clean presets for llm, prompt, evaluator, etc., plus CLI/runtime overrides and environment interpolation.
• Reduces user error and simplifies experimentation and reproducibility.

Minimal implementation

from typing import Optional, Union
from pathlib import Path
import os
from omegaconf import OmegaConf

def load_config(config_path: Optional[Union[str, Path]] = None) -> Config:
    """Load configuration from a YAML file with optional Hydra composition."""
    if config_path and Path(config_path).exists():
        p = Path(config_path)
        dc = OmegaConf.load(str(p))
        cfg_dict = OmegaConf.to_container(dc, resolve=True)

        # If this is a Hydra selector file, compose using its parent folder.
        if isinstance(cfg_dict, dict) and ("defaults" in cfg_dict or "default" in cfg_dict):
            from hydra import compose, initialize_config_dir
            with initialize_config_dir(config_dir=str(p.parent.resolve()), job_name="compose"):
                dc = compose(config_name=p.stem)
            cfg_dict = OmegaConf.to_container(dc, resolve=True)

        config = Config.from_dict(cfg_dict)
    else:
        config = Config()
        api_key = os.environ.get("OPENAI_API_KEY")
        api_base = os.environ.get("OPENAI_API_BASE", "https://api.openai.com/v1")
        config.llm.update_model_params({"api_key": api_key, "api_base": api_base})

    # Keep existing behavior: propagate system message
    config.llm.update_model_params({"system_message": config.prompt.system_message})
    return config

Backward compatibility

• Plain YAML files still work.
• Env var fallback stays the same.
• Dataclass __post_init__ logic unchanged.