The field of prompt optimization has witnessed a breakthrough with GEPA (Genetic Pareto), a novel approach that uses natural language reflection to optimize prompts for large language models. Based on the research published in “GEPA: Genetic Pareto Prompt Optimization for Large Language Models”.  GEPA is an amazing tools for prompt optimization and new GEPA RAG Adapter contributed by us with RAG GUIDE extends the proven genetic pareto optimization methodology to one of the most important applications of LLMs: Retrieval Augmented Generation (RAG). The recently merged GEPA RAG Adapter brings this powerful optimization methodology to RAG systems, enabling automatic optimization of the entire RAG pipeline across multiple vector databases.

Background: The Challenge of RAG Optimization

Retrieval Augmented Generation(RAG)) systems have become essential for building AI applications that need to access and reason over specific knowledge bases. However, optimizing RAG systems has traditionally been a manual, time intensive process requiring domain expertise and extensive trial and error experimentation. Each component of the RAG pipeline from query reformulation to answer generation requires carefully crafted prompts that often need to be tuned separately, making it difficult to achieve optimal end to end performance. The introduction of GEPA’s RAG Adapter addresses this challenge by applying the proven genetic pareto optimization methodology specifically to RAG systems, enabling automatic discovery of optimal prompts across the entire pipeline.

What is GEPA?

GEPA (Genetic Pareto) is a prompt optimization technique for large language models that represents a significant advancement over traditional approaches. The methodology introduces several key innovations:

Natural Language Reflection: Unlike traditional reinforcement learning methods that rely on scalar rewards, GEPA uses natural language as its learning medium. The system samples system level trajectories (including reasoning, tool calls, and outputs), reflects on these trajectories in natural language, diagnoses problems, and proposes prompt updates.

Pareto Frontier Optimization: GEPA maintains a “Pareto frontier” of optimization attempts, combining lessons learned from multiple approaches rather than focusing on a single optimization path. This approach enables more robust and comprehensive optimization.

GEPA demonstrates remarkable efficiency in the research paper, achieving:

• 10% average improvement over Group Relative Policy Optimization (GRPO)
• Up to 20% improvement in best cases
• 35x fewer rollouts compared to traditional methods
• Over 10% improvement compared to leading prompt optimizer MIPROv2

Why GEPA Works for RAG

The interpretable, natural language based approach of GEPA is particularly well suited for RAG optimization because:

1. Complex Interaction Understanding: RAG systems involve complex interactions between retrieval quality and generation quality. GEPA’s natural language reflection can identify and articulate these nuanced relationships.
2. Multi Component Optimization: RAG pipelines require optimizing multiple components simultaneously. GEPA’s Pareto frontier approach can balance trade offs between different components effectively.
3. Interpretable Improvements: The natural language reflection mechanism provides clear insights into why certain prompt modifications improve performance, making the optimization process more transparent and debuggable.

Prompt Optimization with GEPA

GEPA’s prompt optimization process follows a systematic approach that has been proven effective across various LLM applications:

The Optimization Loop

The optimization process consists of six key steps:

1. Trajectory Sampling: GEPA samples complete execution trajectories from the system, capturing not just final outputs but the entire reasoning process.
2. Natural Language Reflection: The system analyzes these trajectories using natural language, identifying patterns, problems, and opportunities for improvement.
3. Diagnostic Analysis: Problems are diagnosed in interpretable terms, such as “query reformulation is too narrow” or “context synthesis includes irrelevant information.”
4. Prompt Proposal: Based on the analysis, GEPA proposes specific prompt modifications using natural language reasoning.
5. Testing and Evaluation: Proposed changes are tested against evaluation criteria, with results fed back into the optimization loop.
6. Pareto Frontier Update: Successful improvements are incorporated into the Pareto frontier, building a comprehensive understanding of what works.

This approach leverages the language understanding capabilities of LLMs themselves to drive the optimization process, creating a self improving system that can articulate and reason about its own performance.

RAG Introduction: The Challenge of Knowledge Retrieval

Retrieval Augmented Generation represents a shift in how we build knowledge intensive AI applications. Traditional language models are limited to the knowledge they were trained on, which becomes outdated and cannot include private or domain specific information. RAG solves this by combining the reasoning capabilities of LLMs with real time access to relevant documents from vector databases.

The RAG Pipeline

A typical RAG system involves several critical steps:

1. Query Processing: User queries must be processed and potentially reformulated to improve retrieval effectiveness.
2. Document Retrieval: Relevant documents are retrieved from a vector database using semantic similarity or hybrid search methods.
3. Document Reranking: Retrieved documents may be reordered based on relevance criteria specific to the query.
4. Context Synthesis: Multiple retrieved documents are synthesized into coherent context that supports answer generation.
5. Answer Generation: The LLM generates a final answer based on the synthesized context and original query.

Each of these steps involves prompts that significantly impact the overall system performance, making optimization crucial for real world applications.

RAG Optimization with GEPA

The GEPA RAG Adapter brings systematic optimization to every component of the RAG pipeline. Here’s how GEPA’s methodology applies to RAG optimization:

Vector Store Agnostic Design

One of the most powerful aspects of the GEPA RAG Adapter is its vector store agnostic design. The adapter provides a unified optimization interface that works across multiple vector databases.

Supported Vector Stores

The adapter supports five major vector databases:

• ChromaDB: Ideal for local development and prototyping. Simple setup with no external dependencies required.
• Weaviate: Production ready with hybrid search capabilities and advanced features. Requires Docker.
• Qdrant: High performance with advanced filtering and payload search capabilities. Can run in memory mode.
• LanceDB: Serverless, developer friendly architecture built on Apache Arrow. No Docker required.
• Milvus: Cloud native scalability with Milvus Lite for local development. No Docker required for Lite mode.

Data Structure for RAG Optimization

The RAG adapter uses a specific data structure for training and validation examples:

train_data = [
    RAGDataInst(
        query="What is machine learning?",
        ground_truth_answer="Machine Learning is a method of data analysis that automates analytical model building. It is a branch of artificial intelligence based on the idea that systems can learn from data, identify patterns and make decisions with minimal human intervention.",
        relevant_doc_ids=["ml_basics"],
        metadata={"category": "definition", "difficulty": "beginner"},
    ),
    RAGDataInst(
        query="How does deep learning work?",
        ground_truth_answer="Deep Learning is a subset of machine learning based on artificial neural networks with representation learning. It can learn from data that is unstructured or unlabeled. Deep learning models are inspired by information processing patterns found in biological neural networks.",
        relevant_doc_ids=["dl_basics"],
        metadata={"category": "explanation", "difficulty": "intermediate"},
    ),
    RAGDataInst(
        query="What is natural language processing?",
        ground_truth_answer="Natural Language Processing (NLP) is a branch of artificial intelligence that helps computers understand, interpret and manipulate human language. NLP draws from many disciplines, including computer science and computational linguistics.",
        relevant_doc_ids=["nlp_basics"],
        metadata={"category": "definition", "difficulty": "intermediate"},
    ),
]

Initial Prompt Templates

The actual implementation includes this initial prompt template for optimization:

# Create initial prompt 
initial_prompts = {
    "answer_generation": """You are an AI expert providing accurate technical explanations.

Based on the retrieved context, provide a clear and informative answer to the user's question.

Guidelines:
- Use information from the provided context
- Be accurate and concise
- Include key technical details
- Structure your response clearly

Context: {context}

Question: {query}

Answer:"""
}

Running GEPA Optimization

The actual optimization call in the working codebase:

# call GEPA Optimization
result = gepa.optimize(
    seed_candidate=initial_prompts,
    trainset=train_data,
    valset=val_data,
    adapter=rag_adapter,
    reflection_lm=llm_client,
    max_metric_calls=args.max_iterations,
)

# Accessing results
best_score = result.val_aggregate_scores[result.best_idx]
optimized_prompts = result.best_candidate
total_iterations = result.total_metric_calls

Implementation and Usage

# Base installation
pip install gepa

# Vector store dependencies
pip install chromadb                          # ChromaDB
pip install lancedb pyarrow sentence-transformers  # LanceDB
pip install pymilvus sentence-transformers    # Milvus
pip install qdrant-client                     # Qdrant  
pip install weaviate-client                   # Weaviate

Using the Unified Optimization Script

The GEPA repository includes a working unified script with these actual command line options:

# Navigate to the actual examples directory
cd src/gepa/examples/rag_adapter


# ChromaDB (default, no external dependencies)
python rag_optimization.py --vector-store chromadb

# LanceDB (local, no Docker required)
python rag_optimization.py --vector-store lancedb

# Milvus Lite (local SQLite based)
python rag_optimization.py --vector-store milvus

# Qdrant (in memory or with Docker)
python rag_optimization.py --vector-store qdrant

# Weaviate (requires Docker)
python rag_optimization.py --vector-store weaviate

# With specific models (actual model names from the code)
python rag_optimization.py --vector-store chromadb --model ollama/llama3.1:8b

# Full optimization run
python rag_optimization.py --vector-store qdrant --max-iterations 20

# Test setup without optimization
python rag_optimization.py --vector-store chromadb --max-iterations 0

Command Line Arguments

From the actual argument parser in the code:

# These are command line arguments implemented
parser.add_argument(
    "--vector-store",
    type=str,
    default="chromadb",
    choices=["chromadb", "lancedb", "milvus", "qdrant", "weaviate"],
    help="Vector store to use (default: chromadb)"
)
parser.add_argument(
    "--model",
    type=str,
    default="ollama/qwen3:8b",
    help="LLM model (default: ollama/qwen3:8b)"
)
parser.add_argument(
    "--embedding-model",
    type=str,
    default="ollama/nomic-embed-text:latest",
    help="Embedding model (default: ollama/nomic-embed-text:latest)",
)
parser.add_argument(
    "--max-iterations",
    type=int,
    default=5,
    help="GEPA optimization iterations (default: 5, use 0 to skip optimization)",
)
parser.add_argument("--verbose", action="store_true", help="Enable verbose output")

Features and Capabilities

Multi Component Optimization

The GEPA RAG Adapter optimizes prompts for four key components (though the current implementation focuses primarily on answer generation in the initial prompts):

1. Query Reformulation: Transforms user queries to improve retrieval effectiveness
2. Context Synthesis: Combines retrieved documents into coherent context
3. Answer Generation: Produces final answers based on synthesized context
4. Document Reranking: Reorders retrieved documents by relevance

Evaluation System

The adapter includes comprehensive evaluation that measures both retrieval and generation quality:

# From the actual evaluation call
eval_result = rag_adapter.evaluate(
    batch=val_data[:1], 
    candidate=initial_prompts, 
    capture_traces=True
)

# Accessing evaluation results
initial_score = eval_result.scores[0]
sample_answer = eval_result.outputs[0]['final_answer']

RAG Configuration

The actual RAG configuration options:

# These are the configuration options used 
rag_config = {
    "retrieval_strategy": "similarity",
    "top_k": 3,
    "retrieval_weight": 0.3,
    "generation_weight": 0.7,
}

# For Weaviate with hybrid search 
if args.vector_store == "weaviate":
    rag_config["retrieval_strategy"] = "hybrid"
    rag_config["hybrid_alpha"] = 0.7

Quick Start

The best to get started with RAG Optimization using GEPA is to refer the GEPA_GUIDE from the repository. It has all the instruction to get started and run examples with all the details.

Prerequisites and Setup

You can try this locally using the following Ollama models if your system can run them.

For Ollama Models:

# These are the actual Ollama model requirements
ollama pull qwen3:8b
ollama pull nomic-embed-text:latest

Get you Ollama models and relevant dependency (chromed) and run this from GEPA repo as quick start.

# Quick start with ChromaDB
cd src/gepa/examples/rag_adapter
python rag_optimization.py --vector-store chromadb --max-iterations 10

For Weaviate (actual Docker command):

# This is the actual Docker command from the documentation
docker run -p 8080:8080 -p 50051:50051 cr.weaviate.io/semitechnologies/weaviate:1.26.1

For Qdrant (optional Docker setup):

# Optional Qdrant Docker setup
docker run -p 6333:6333 qdrant/qdrant

Watch Demo

Summary

The GEPA RAG Adapter represents an advancement in RAG system optimization, bringing the proven genetic pareto methodology to one of the most important applications of large language models. Key benefits include:

Technical Advantages

• Automated Optimization: Eliminates manual prompt engineering for RAG systems using GEPA’s natural language reflection approach
• Vector Store Agnostic: Works across ChromaDB, Weaviate, Qdrant, Milvus, and LanceDB with the same interface
• Efficiency: Leverages GEPA’s proven efficiency gains (35x fewer rollouts than traditional methods)
• Interpretable Process: Natural language reflection provides insights into optimization decisions

Potential Benefits

• Unified Interface: Single script works across all supported vector stores
• Flexible Deployment: Supports both local (Ollama) and cloud models
• Production Ready: Graceful dependency handling and error management
• Extensible Design: Easy to add new vector stores through the interface

Scientific Foundation

• Research Backed: Based on peer reviewed research demonstrating GEPA’s effectiveness
• Natural Language Reflection: Uses interpretable optimization that provides insights into improvements
• Pareto Frontier Optimization: Maintains multiple optimization paths for robust performance

Conclusion

The integration of GEPA’s genetic pareto optimization methodology with RAG systems is still early but good start. As of the now best use to GEPA is with DSPy adapters but you can optimize your RAG pipelines using standalone GEPA as well if you don’t have DSPy in your tech stack. By applying the proven GEPA approach which uses natural language reflection and Pareto frontier optimization to the complex challenge of RAG system optimization, developers now have access to a systematic, automated approach for building high performance knowledge retrieval systems.The GEPA RAG Adapter addresses the technical challenges of multi component optimization in a way that is interpretable, efficient, and adaptable to different deployment requirements. The unified script enables easy experimentation across different vector stores, while the vector store agnostic design ensures that optimization work translates across different deployment environments.

The GEPA RAG Adapter is available today in the GEPA repository, with working examples and comprehensive documentation to get you started immediately.