🧬 Evo-Prompt

Evo-Prompt is an automated evolutionary loop designed for high-sensitivity image generation. While standard LLMs can handle basic prompts, specialized diffusion pipelines—particularly those utilizing LoRAs or fine-tuned models—are extremely sensitive to phrasing. Evo-Prompt solves this by automatically discovering the precise prompt structures required to trigger successful results in these fragile, complex workflows.

---

⚙️ How It Works

Evo-Prompt operates on a continuous evolutionary cycle:

1. Expand (Diversity): The LLM takes a raw user intent and generates a diverse population of initial prompts.
2. Generate (Diffusion): The prompts are sent to a local ComfyUI instance to generate a batch of images.
3. Evaluate (Vision): A Vision-Language Model (served via a local llama.cpp container) acts as a judge, scoring each image against a multi-dimensional rubric (Fidelity, Composition, Style, etc.).
4. Mutate (Refinement): The best-performing prompts and their specific critiques are fed back to the LLM. The LLM then "mutates" the prompts—fixing flaws and doubling down on strengths—to produce a superior next generation.

---

🚀 Getting Started

Prerequisites

• Docker with NVIDIA Container Toolkit installed for GPU passthrough.
• ComfyUI: Models & custom nodes ready locally (see Workflow Setup).
• uv: A high-performance Python package and project manager.
• GPU: NVIDIA GPU with sufficient VRAM to run the LLM, embedding, and diffusion models.

Installation

1. Clone the repository:

   git clone https://github.com/alantsov/evo-prompt.git
   cd evo-prompt

2. Install dependencies:

   uv sync

💡 Model Management: LLM and embedding models are automatically downloaded to ~/.cache/huggingface on first run via the llama.cpp containers. No manual pulling is required.

Usage

Single Intent:

uv run evoprompt -p "anime girl repairing drone in garage, cyberpunk style"

Batch Intents (from file): Add one intent per line to prompts.md, then run:

uv run evoprompt -f prompts.md

Advanced Configuration:

uv run evoprompt \
  -p "cyberpunk cityscape at night" \
  -w 8 \
  -g 6 \
  -k 4 \
  -n 3 \
  -a gepa \
  -l debug \
  -y krea-2-advanced.json

Flag	Description	Default
-p, --prompt	Single intent string to optimize	(required if no file)
-f, --file	Path to file with intents (one per line)	prompts.md
-w, --pop-size	Population size per generation	4
-g, --generations	Number of optimization generations	4
-k, --top-k	Top-K parents to keep for mutation/crossover	4
-n, --top-images	Number of best images to save to output dir	1
-a, --optimizer	Algorithm: gepa (diversity-focused) or beam (exploitation-focused)	gepa
-y, --workflow	ComfyUI workflow JSON template	krea-2-basic.json
-l, --log-level	Logging verbosity (debug, info, warning, error)	info
-o, --output-dir	Directory to save final images & logs	output
-c, --config	Path to configuration YAML file	config.yml

---

🛠 Workflow Setup

To use the provided Krea-2 workflow, you must have the following models placed in your ComfyUI directory:

Model Type	Filename	ComfyUI Directory	HF repository
Diffusion (UNET)	krea2_turbo_bf16.safetensors	ComfyUI/models/diffusion_models/	Download
CLIP	qwen3vl_4b_bf16.safetensors	ComfyUI/models/text_encoders/	Download
VAE	qwen_image_vae.safetensors	ComfyUI/models/vae/	Download
LoRA	your_lora.safetensors	ComfyUI/models/loras/	your lora

download them manually or use script:

uv run evoprompt-pull

---

🧪 Testing

The project includes a suite of basic unit tests. To run the tests, use:

uv run pytest

---

📂 Project Structure

• evo_prompt/: Core logic (Optimization loop, ComfyUI/llama.cpp wrappers, LLM managers).
• workflows/: JSON workflow templates for ComfyUI.
• data/: Stores all generated images and the optimization trajectory (JSON logs of every generation).
• output/: Contains the final "Best" image produced by the loop.
• config.yml: Central configuration for models, prompts, and rules.

---

🛠 Configuration

The config.yml file controls the "brain" of the agent:

• models: Define which HuggingFace GGUF models to use for optimization, evaluation, and embeddings. Models are auto-cached in ~/.cache/huggingface.
• prompts: Customize the system prompts used by the Evaluator and the Optimizer to fine-tune the behavior of the loop.
• comfyui: Define the API endpoint and the specific workflow JSON file to use.
• rubric_weights: Adjust the importance of each evaluation criterion.

---

⚖️ License & Model Disclaimer

This project's source code is licensed under the MIT License.

Third-Party Model Weights The MIT license does not cover the Krea 2 diffusion model, CLIP, VAE, LoRA weights, or any other third-party assets referenced in this repository. These models are developed and distributed by their respective creators. By using this project, you acknowledge that:

• You are solely responsible for legally acquiring the required model weights.
• You must comply with all terms, licenses, and usage restrictions imposed by the original model publishers.
• The project author provides no warranty, support, or liability regarding third-party models, their outputs, or their compliance with local regulations.

Please review the official licensing terms of any models you integrate with this pipeline.