MLflow Prompt Optimization
mlflow.org
- Free plan
- Web
- API
- Self-hosted
What MLflow Prompt Optimization is
MLflow Prompt Optimization automates prompt refinement by evaluating prompts against data, finding failure patterns, and generating improved variants iteratively. Its `mlflow.genai.optimize_prompts` API provides a common interface for optimization algorithms, including GEPA and Metaprompting. You supply training data and scorers, with options to define custom scorers and aggregation functions. Optimized prompts can be saved as new Prompt Registry versions, while runs, metrics, and traces can be tracked for comparison and rollback. The workflow supports LangChain, LangGraph, OpenAI Agent, Pydantic AI, CrewAI, AutoGen, and custom frameworks, and the product page says it works with any LLM provider. The software is Apache 2.0 licensed, with a free open-source option that can be self-hosted or managed through cloud providers. Self-hosted users are offered community support. Documentation recommends GEPA for tasks with clear evaluation metrics where quality is critical, citing medical and financial agents as examples. Optimization costs depend on the reflection model and the maximum number of metric calls. MLflow documents basic HTTP authentication for tracking-server resources, including prompts, and security middleware from version 3.5.0 onward for several network risks.
Who it suits
It suits teams that want to optimize prompts against their own data and track versions, runs, and evaluation results. GEPA may fit tasks with clear metrics where quality is critical, such as medical or financial agents.
Good to know, and what to watch
- Supports GEPA and Metaprompting algorithms.
- Users can define custom scorers and aggregation functions.
- Optimized prompts can be versioned and rolled back.
- Works with listed frameworks and any LLM provider.
- Apache 2.0 licensed with a free open-source option.
- Optimization cost depends on model and metric-call limit.
- Self-hosted support is community-based.
- GEPA is best suited to datasets of 100 or more records.
Plans
Every plan and priceMLflow Prompt Optimization (open source)
Free plan
The verdict
MLflow Prompt Optimization provides an open-source workflow for evaluating and iterating on prompts, with version tracking for review and rollback. Plan for optimization costs, which vary with the reflection model and metric-call limit.
What the maker publishes
- Algorithms
- The documentation lists GEPA and Metaprompting as supported optimization algorithms. source
- Data guidance
- The product page's example recommends 50–100 labeled training examples; the documentation says GEPA is best suited to a dataset of 100 or more records. source
- Evaluation
- Users can supply scorers and training data, and can define custom scorers and aggregation functions. source
- Framework integrations
- The optimization workflow works with LangChain, LangGraph, OpenAI Agent, Pydantic AI, CrewAI, AutoGen, or custom frameworks. source
- License and governance
- MLflow is licensed under Apache 2.0 and is backed by the Linux Foundation. source
- Network security
- MLflow 3.5.0 and later includes tracking-server security middleware for DNS rebinding, CORS, clickjacking, and security headers. source
- Optimization API
- The `mlflow.genai.optimize_prompts` API provides a common interface for prompt optimization algorithms. source
- Optimization cost
- The documentation says GEPA optimization cost depends on the reflection model and the maximum number of metric calls. source
- Prompt versioning
- Optimized prompts can be saved as new Prompt Registry versions, and runs, metrics, and traces can be tracked for comparison and rollback. source
- Provider support
- The product page says the workflow works with any LLM provider. source
- Purpose
- Automates prompt engineering by evaluating prompts on data, identifying failure patterns, and iteratively generating improved variants. source
- Security controls
- MLflow documents basic HTTP authentication with permissions for tracking-server resources, including prompts. source
- Support
- The self-hosted open-source option lists community support. source
- Use case fit
- The documentation recommends GEPA for tasks with clear evaluation metrics and where quality is critical, citing medical and financial agents as examples. source
Checked against the maker's pages on 4 Oct 2026.