PromptWizard
microsoft.github.io
- Free plan
- Windows
- macOS
- Linux
- API
- Self-hosted
What PromptWizard is
PromptWizard is a free, MIT-licensed open-source framework for optimizing prompts and examples through an iterative feedback process. It creates prompt variations, scores and critiques them, then refines the instructions. It can also optimize in-context examples and synthesize examples suited to a task. The project describes use without examples, with synthetic examples, or with training data. Listed datasets include GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII). Custom datasets use JSONL samples with question and answer fields, and require task-specific answer extraction, evaluation functions, configuration, and data files. PromptWizard can generate chain-of-thought reasoning for examples; this can be switched off to reduce prompt size. Installation is through its GitHub repository as a Python package in development mode, with instructions for Windows, macOS, and Linux. Model access setup supports OpenAI API keys and Azure OpenAI endpoints, so credentials for those services are required. The README reports average optimization times of around 20–30 minutes in experiments on its listed datasets, with timing dependent on the dataset, and says human supervision can help tune generated prompts.
Who it suits
PromptWizard suits developers and teams looking to automate prompt refinement or example generation. Custom-dataset users should be prepared to supply task-specific evaluation functions and configuration.
Good to know, and what to watch
- Free and MIT licensed.
- Optimizes prompts and in-context examples.
- Supports Windows, macOS, and Linux installation.
- Offers an option to disable reasoning generation.
- Model access requires OpenAI or Azure OpenAI credentials.
- Custom datasets need task-specific evaluation functions.
- Prompt tuning can benefit from user supervision.
Plans
Every plan and priceMIT-licensed open-source software
Free plan
Getting started
- Open the PromptWizard project website or GitHub repository
- Create a virtual environment on Windows, macOS, or Linux
- Install the Python package from the repository in development mode
- Prepare supported data or JSONL samples with question and answer fields
- Configure OpenAI API keys or an Azure OpenAI endpoint
- Run iterative prompt and example optimization
Limits to check
The free plan has no listed software charge, but model access depends on OpenAI or Azure OpenAI credentials. Custom datasets require their own extraction, evaluation, configuration, and data files.
The Club's review
PromptWizard is a free, MIT-licensed Python framework for developers and researchers who want automated, task-aware improvement of prompts and in-context examples. It is best suited to people comfortable with datasets, evaluation code, and model credentials; its main appeal is iterative refinement rather than a hosted prompt-writing interface.
Overview
PromptWizard runs a feedback loop that creates prompt variants, scores them, examines their successes and failures, and revises the instructions. It can improve examples at the same time, including generating varied examples tailored to a task. That makes it useful when prompt quality needs to be judged against actual work rather than by wording preference alone.
The framework covers workflows with no examples, synthetic examples, or existing training data. It can generate chain-of-thought reasoning for in-context examples, while a configuration option turns that generation off when shorter prompts or lower token use are more important. Generated prompts are often detailed, so supervision is worthwhile before adopting them unchanged.
Key features
Iterative prompt and example optimization
Generation, scoring, critique, and refinement are combined in repeated iterations. Optimizing demonstrations as well as instructions is a meaningful advantage for tasks whose performance depends on examples. Readers seeking a quick one-off suggestion, however, may find this development-oriented loop excessive.
Dataset-based evaluation
The README names GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII) as supported datasets. Custom data uses JSONL samples with question and answer fields. Bringing another dataset into the workflow requires answer-extraction and evaluation functions plus configuration and data files, so custom use demands engineering effort rather than just an upload.
Model connectivity and reasoning control
Setup supports OpenAI API keys and Azure OpenAI endpoints, and the framework is described as supporting multiple models. Chain-of-thought generation can enrich in-context examples, but disabling it provides a direct way to control prompt size and token consumption. API credentials remain the user's responsibility.
Optimization speed and oversight
Experiments on the listed datasets took about 20–30 minutes on average, with duration dependent on the dataset. That is practical for iterative experiments, not a promise of a fixed turnaround. Human review is recommended because detailed generated instructions may still need task-specific tuning.
Pricing
| Plan | Price | What it means |
|---|---|---|
| MIT-licensed open-source software | Free | All software is free; no paid plans are listed. Model API access requires OpenAI or Azure OpenAI credentials. |
The free plan suits engineers, researchers, and organizations able to operate their own Python environment and manage model usage costs. It has no subscription charge, but API calls are not bundled, and custom-dataset preparation can be the larger cost in time and labor.
Platforms
PromptWizard supports API, Linux, macOS, self-hosted, and Windows use. Installation instructions cover virtual environments on all three desktop operating systems and install the package from its GitHub repository in development mode. This is a fit for teams that control their runtime; it is not positioned as a browser-only service.
Who it's for
Choose PromptWizard if you need repeatable prompt experiments against benchmark or custom data, want examples optimized alongside instructions, and have Python and evaluation capability in-house. It is particularly relevant to researchers and application teams comparing prompt variants across tasks.
Look elsewhere if you want a hosted editor, an immediate no-code workflow, bundled model access, or a turnkey evaluation service. The documented setup involves development-mode installation, credentials, dataset files, and custom evaluation work when your data falls outside the named datasets.
Pros and cons
- Pro: Feedback-driven iterations connect prompt changes to scores, giving optimization a task-based basis.
- Pro: Instructions and in-context examples can be refined together, including synthetic examples.
- Pro: MIT licensing and free software remove subscription fees and support self-hosting.
- Con: Development-mode Python installation and model credentials raise the setup bar for nontechnical users.
- Con: Custom datasets require extraction, evaluation, configuration, and data files.
- Con: Generated prompts may be lengthy and still need human tuning; optimization time varies by dataset.
Alternatives
Promptitude is the better choice for a hosted API and web product with a freemium plan, free trial, and explicit generation and tool-call quotas.
Google Cloud Agent Evaluation fits teams wanting a paid, web-and-API evaluation service with a free trial and usage-based metric pricing.
Vellum suits readers seeking a freemium product spanning web and desktop platforms, with a free starting plan and paid credits.
VantagePrompt is a web-based freemium option with a small free monthly credit allowance and request limits.
Agenta is worth considering for self-hosted or web agent workflows with free team and run quotas.
ChainForge is another free, open-source self-hosted and web option.
GEPA is a free MIT-licensed open-source Python package installable with pip.
Latitude offers a freemium API, self-hosted, and web product with a free credit allocation and a paid Pro tier.
For a wider list, see AI Prompt Generators.
Verdict
PromptWizard is a strong choice for technical teams that want free, self-hosted, data-driven prompt and example optimization, especially when benchmark or custom evaluation matters. Choose it for the iterative, task-aware feedback loop; look elsewhere if development setup, API billing, and evaluation engineering outweigh the value of controlling the workflow.
The verdict
PromptWizard offers an iterative way to refine instructions and examples, with support for listed datasets and custom data. Its setup involves Python development-mode installation and model-service credentials.
Questions people ask
Is PromptWizard free?
Yes. It is offered as free, MIT-licensed open-source software. API access still requires OpenAI or Azure OpenAI credentials.
Which platforms support installation?
Installation instructions cover Windows, macOS, and Linux, and the project lists API and self-hosted use.
What datasets are supported?
The README lists GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII). Custom datasets use JSONL question-and-answer samples.
Can it optimize examples as well as prompts?
Yes. It optimizes in-context examples and can synthesize diverse, task-relevant examples.
Can reasoning generation be turned off?
Yes. Chain-of-thought generation for in-context examples can be disabled to reduce prompt length or token count.
What the maker publishes
- Custom data requirements
- Custom datasets are expected in JSONL format with question and answer fields for each sample. source
- Customization
- Custom datasets require dataset-specific answer extraction and evaluation functions, along with configuration and data files. source
- Dataset format
- Custom datasets are expected as JSONL files with question and answer fields in each sample. source
- Dataset support
- The README lists GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII) as supported training datasets. source
- Example optimization
- It optimizes in-context examples alongside prompt instructions and can synthesize diverse, task-relevant examples. source
- Example optimization
- It optimizes prompt instructions and few-shot examples together, including by synthesizing diverse, task-relevant examples. source
- Founded
- 1975 source
- Headquarters
- Redmond, Washington, USA source
- Human review
- The README says generated prompts are usually detailed and that user supervision can help tune them for the task. source
- Human supervision
- The README says generated prompts are usually detailed and that user supervision can help tune them further for a task. source
- Installation
- The project is installed from its GitHub repository as a Python package in development mode, with setup instructions for Windows, macOS, and Linux. source
- Installation platforms
- Installation instructions cover virtual environments on Windows, macOS, and Linux and package installation in development mode. source
- License
- The repository identifies the project as MIT licensed. source
- License
- The repository includes an MIT License granting permission to use, copy, modify, distribute, sublicense, and sell copies subject to its terms. source
- Maker
- The project page names Microsoft Research and lists Eshaan Agarwal, Joykirat Singh, Vivek Dani, Raghav Magazine, Tanuja Ganu, and Akshay Nambi as authors. source
- Model API integrations
- The setup instructions support OpenAI API keys and Azure OpenAI endpoints for LLM access. source
- Model API integrations
- The README says the code requires LLM access through API calls and supports Azure endpoints or OpenAI keys. source
- Optimization time
- The README says optimization took around 20–30 minutes on average in its experiments on the listed datasets, with time depending on the dataset. source
- Optimization time
- The README reports that optimization took around 20–30 minutes on average in experiments on the listed datasets. source
- Product
- PromptWizard is an open source framework for automated prompt and example optimization using a feedback-driven critique and synthesis process. source
- Prompt optimization
- It iteratively generates, scores, critiques, and refines prompt instructions. source
- Prompt refinement
- It generates prompt variations, scores them, critiques their successes and failures, and refines prompts over iterations. source
- Purpose
- PromptWizard is an open-source framework for automated, task-aware prompt and example optimization. source
- Reasoning
- It can generate chain-of-thought reasoning for in-context examples, and this option can be disabled to reduce prompt length or token count. source
Checked against the maker's pages on 3 Oct 2026.