Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

PromptWizard

microsoft.github.io

  • Free plan
  • Windows
  • macOS
  • Linux
  • API
  • Self-hosted

What PromptWizard is

PromptWizard is a free, MIT-licensed open-source framework for optimizing prompts and examples through an iterative feedback process. It creates prompt variations, scores and critiques them, then refines the instructions. It can also optimize in-context examples and synthesize examples suited to a task. The project describes use without examples, with synthetic examples, or with training data. Listed datasets include GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII). Custom datasets use JSONL samples with question and answer fields, and require task-specific answer extraction, evaluation functions, configuration, and data files. PromptWizard can generate chain-of-thought reasoning for examples; this can be switched off to reduce prompt size. Installation is through its GitHub repository as a Python package in development mode, with instructions for Windows, macOS, and Linux. Model access setup supports OpenAI API keys and Azure OpenAI endpoints, so credentials for those services are required. The README reports average optimization times of around 20–30 minutes in experiments on its listed datasets, with timing dependent on the dataset, and says human supervision can help tune generated prompts.

Who it suits

PromptWizard suits developers and teams looking to automate prompt refinement or example generation. Custom-dataset users should be prepared to supply task-specific evaluation functions and configuration.

Good to know, and what to watch

  • Free and MIT licensed.
  • Optimizes prompts and in-context examples.
  • Supports Windows, macOS, and Linux installation.
  • Offers an option to disable reasoning generation.
  • Model access requires OpenAI or Azure OpenAI credentials.
  • Custom datasets need task-specific evaluation functions.
  • Prompt tuning can benefit from user supervision.

MIT-licensed open-source software

Free

Free plan

Getting started

  1. Open the PromptWizard project website or GitHub repository
  2. Create a virtual environment on Windows, macOS, or Linux
  3. Install the Python package from the repository in development mode
  4. Prepare supported data or JSONL samples with question and answer fields
  5. Configure OpenAI API keys or an Azure OpenAI endpoint
  6. Run iterative prompt and example optimization

Limits to check

The free plan has no listed software charge, but model access depends on OpenAI or Azure OpenAI credentials. Custom datasets require their own extraction, evaluation, configuration, and data files.

The Club's review

PromptWizard is a free, MIT-licensed Python framework for developers and researchers who want automated, task-aware improvement of prompts and in-context examples. It is best suited to people comfortable with datasets, evaluation code, and model credentials; its main appeal is iterative refinement rather than a hosted prompt-writing interface.

Overview

PromptWizard runs a feedback loop that creates prompt variants, scores them, examines their successes and failures, and revises the instructions. It can improve examples at the same time, including generating varied examples tailored to a task. That makes it useful when prompt quality needs to be judged against actual work rather than by wording preference alone.

The framework covers workflows with no examples, synthetic examples, or existing training data. It can generate chain-of-thought reasoning for in-context examples, while a configuration option turns that generation off when shorter prompts or lower token use are more important. Generated prompts are often detailed, so supervision is worthwhile before adopting them unchanged.

Key features

Iterative prompt and example optimization

Generation, scoring, critique, and refinement are combined in repeated iterations. Optimizing demonstrations as well as instructions is a meaningful advantage for tasks whose performance depends on examples. Readers seeking a quick one-off suggestion, however, may find this development-oriented loop excessive.

Dataset-based evaluation

The README names GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII) as supported datasets. Custom data uses JSONL samples with question and answer fields. Bringing another dataset into the workflow requires answer-extraction and evaluation functions plus configuration and data files, so custom use demands engineering effort rather than just an upload.

Model connectivity and reasoning control

Setup supports OpenAI API keys and Azure OpenAI endpoints, and the framework is described as supporting multiple models. Chain-of-thought generation can enrich in-context examples, but disabling it provides a direct way to control prompt size and token consumption. API credentials remain the user's responsibility.

Optimization speed and oversight

Experiments on the listed datasets took about 20–30 minutes on average, with duration dependent on the dataset. That is practical for iterative experiments, not a promise of a fixed turnaround. Human review is recommended because detailed generated instructions may still need task-specific tuning.

Pricing

PlanPriceWhat it means
MIT-licensed open-source softwareFreeAll software is free; no paid plans are listed. Model API access requires OpenAI or Azure OpenAI credentials.

The free plan suits engineers, researchers, and organizations able to operate their own Python environment and manage model usage costs. It has no subscription charge, but API calls are not bundled, and custom-dataset preparation can be the larger cost in time and labor.

Platforms

PromptWizard supports API, Linux, macOS, self-hosted, and Windows use. Installation instructions cover virtual environments on all three desktop operating systems and install the package from its GitHub repository in development mode. This is a fit for teams that control their runtime; it is not positioned as a browser-only service.

Who it's for

Choose PromptWizard if you need repeatable prompt experiments against benchmark or custom data, want examples optimized alongside instructions, and have Python and evaluation capability in-house. It is particularly relevant to researchers and application teams comparing prompt variants across tasks.

Look elsewhere if you want a hosted editor, an immediate no-code workflow, bundled model access, or a turnkey evaluation service. The documented setup involves development-mode installation, credentials, dataset files, and custom evaluation work when your data falls outside the named datasets.

Pros and cons

  • Pro: Feedback-driven iterations connect prompt changes to scores, giving optimization a task-based basis.
  • Pro: Instructions and in-context examples can be refined together, including synthetic examples.
  • Pro: MIT licensing and free software remove subscription fees and support self-hosting.
  • Con: Development-mode Python installation and model credentials raise the setup bar for nontechnical users.
  • Con: Custom datasets require extraction, evaluation, configuration, and data files.
  • Con: Generated prompts may be lengthy and still need human tuning; optimization time varies by dataset.

Alternatives

Promptitude is the better choice for a hosted API and web product with a freemium plan, free trial, and explicit generation and tool-call quotas.

Google Cloud Agent Evaluation fits teams wanting a paid, web-and-API evaluation service with a free trial and usage-based metric pricing.

Vellum suits readers seeking a freemium product spanning web and desktop platforms, with a free starting plan and paid credits.

VantagePrompt is a web-based freemium option with a small free monthly credit allowance and request limits.

Agenta is worth considering for self-hosted or web agent workflows with free team and run quotas.

ChainForge is another free, open-source self-hosted and web option.

GEPA is a free MIT-licensed open-source Python package installable with pip.

Latitude offers a freemium API, self-hosted, and web product with a free credit allocation and a paid Pro tier.

For a wider list, see AI Prompt Generators.

Verdict

PromptWizard is a strong choice for technical teams that want free, self-hosted, data-driven prompt and example optimization, especially when benchmark or custom evaluation matters. Choose it for the iterative, task-aware feedback loop; look elsewhere if development setup, API billing, and evaluation engineering outweigh the value of controlling the workflow.

The verdict

PromptWizard offers an iterative way to refine instructions and examples, with support for listed datasets and custom data. Its setup involves Python development-mode installation and model-service credentials.

Questions people ask

Is PromptWizard free?

Yes. It is offered as free, MIT-licensed open-source software. API access still requires OpenAI or Azure OpenAI credentials.

Which platforms support installation?

Installation instructions cover Windows, macOS, and Linux, and the project lists API and self-hosted use.

What datasets are supported?

The README lists GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII). Custom datasets use JSONL question-and-answer samples.

Can it optimize examples as well as prompts?

Yes. It optimizes in-context examples and can synthesize diverse, task-relevant examples.

Can reasoning generation be turned off?

Yes. Chain-of-thought generation for in-context examples can be disabled to reduce prompt length or token count.

What the maker publishes

Custom data requirements
Custom datasets are expected in JSONL format with question and answer fields for each sample. source
Customization
Custom datasets require dataset-specific answer extraction and evaluation functions, along with configuration and data files. source
Dataset format
Custom datasets are expected as JSONL files with question and answer fields in each sample. source
Dataset support
The README lists GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII) as supported training datasets. source
Example optimization
It optimizes in-context examples alongside prompt instructions and can synthesize diverse, task-relevant examples. source
Example optimization
It optimizes prompt instructions and few-shot examples together, including by synthesizing diverse, task-relevant examples. source
Founded
1975 source
Headquarters
Redmond, Washington, USA source
Human review
The README says generated prompts are usually detailed and that user supervision can help tune them for the task. source
Human supervision
The README says generated prompts are usually detailed and that user supervision can help tune them further for a task. source
Installation
The project is installed from its GitHub repository as a Python package in development mode, with setup instructions for Windows, macOS, and Linux. source
Installation platforms
Installation instructions cover virtual environments on Windows, macOS, and Linux and package installation in development mode. source
License
The repository identifies the project as MIT licensed. source
License
The repository includes an MIT License granting permission to use, copy, modify, distribute, sublicense, and sell copies subject to its terms. source
Maker
The project page names Microsoft Research and lists Eshaan Agarwal, Joykirat Singh, Vivek Dani, Raghav Magazine, Tanuja Ganu, and Akshay Nambi as authors. source
Model API integrations
The setup instructions support OpenAI API keys and Azure OpenAI endpoints for LLM access. source
Model API integrations
The README says the code requires LLM access through API calls and supports Azure endpoints or OpenAI keys. source
Optimization time
The README says optimization took around 20–30 minutes on average in its experiments on the listed datasets, with time depending on the dataset. source
Optimization time
The README reports that optimization took around 20–30 minutes on average in experiments on the listed datasets. source
Product
PromptWizard is an open source framework for automated prompt and example optimization using a feedback-driven critique and synthesis process. source
Prompt optimization
It iteratively generates, scores, critiques, and refines prompt instructions. source
Prompt refinement
It generates prompt variations, scores them, critiques their successes and failures, and refines prompts over iterations. source
Purpose
PromptWizard is an open-source framework for automated, task-aware prompt and example optimization. source
Reasoning
It can generate chain-of-thought reasoning for in-context examples, and this option can be disabled to reduce prompt length or token count. source

Checked against the maker's pages on 3 Oct 2026.