Promptfoo
promptfoo.dev
- Free plan
- Open source
- Windows
- macOS
- Linux
- Web
- API
- Self-hosted
What Promptfoo is
Promptfoo is a platform for building and evaluating AI applications. Developers can compare prompt results across models in a web view, then use red-team simulations to look for risks such as prompt injection, jailbreaks, data leaks, business-rule violations, and unsafe agent tool use. Its getting-started guide lists 60+ providers, including OpenAI, Anthropic, Google, and local models such as Ollama. Integrations include CI systems, security tools, Google Sheets, and MCP. Promptfoo supports custom metrics, LLM-as-a-judge evaluations, safety checks, human review workflows, and CI/CD integration. The Community plan costs nothing and includes all LLM evaluation features, all model providers and integrations, and 10k red-team probes per month; it can run locally or be self-hosted. Enterprise and On-Premise plans have custom pricing, with different deployment options. Installation is available through npm, npx, or Homebrew. It requires Node.js 22.22.0 or newer and recommends Node.js 24 LTS. The company says it remains open source and is now part of OpenAI.
Who it suits
Promptfoo suits developers and enterprise teams evaluating prompts and checking AI applications for security risks. Its local, self-hosted, managed-cloud, and on-premise options serve teams with different deployment needs.
Good to know, and what to watch
- Compares prompt outputs across multiple models.
- Red-team simulations probe several AI security risks.
- Supports 60+ providers and listed integrations.
- Community includes 10k red-team probes per month.
- Offers local, self-hosted, cloud, and on-premise deployment.
- Requires Node.js 22.22.0 or newer.
- Community red-team use is limited to 10k probes monthly.
- Enterprise and On-Premise pricing is custom and unlisted.
Plans
Every plan and priceEnterprise
On-Premise
Community
Free plan
Getting started
- Open the Promptfoo website
- Install Promptfoo with npm, npx, or Homebrew
- Use Node.js 22.22.0 or newer; Node.js 24 LTS is recommended
- Choose Community for local or self-hosted use, or select Enterprise or On-Premise
- Connect your model providers and listed integrations
Limits to check
Community includes 10k red-team probes per month. Enterprise and On-Premise use custom red-teaming limits.
The Club's review
Promptfoo is an evaluation and security-testing platform for developers building AI applications, from solo practitioners to enterprise teams. It is strongest when teams want to compare model outputs and probe for risks in the same development workflow; the free plan’s 10k monthly probe cap and Node.js requirement are meaningful constraints.
Overview
Promptfoo combines prompt evaluation across models with simulated red-team testing. Evaluations open a web view for comparing outputs; red teaming targets risks including prompt injection, jailbreaks, data leaks, business-rule violations and insecure agent tool use. The company was founded in 2024, is based in San Francisco, and says it is now part of OpenAI while remaining open source.
Key features
Evaluation and provider coverage
Promptfoo supports custom metrics, LLM-as-a-judge, safety evaluations and human review workflows. Its getting-started guide names more than 60 providers, including OpenAI, Anthropic, Google and local models such as Ollama. That breadth is useful for teams comparing models across hosted and local environments, rather than evaluating against just one provider.
Red teaming and integrations
Red-team runs simulate users to look for weaknesses in prompts and agents. Community includes 10k probes per month, with each probe defined as one request to the target system. That offers a substantial starting allowance, but teams running frequent or large-scale tests will need to monitor usage or consider custom limits.
Integrations include GitHub Actions, GitLab CI, Jenkins, Azure Pipelines, CircleCI and Bitbucket Pipelines, alongside Splunk, Burp Suite, Google Sheets and MCP. CI/CD integration can put evaluation into development workflows, while the security and data tools extend connections beyond deployment pipelines.
Installation and deployment
Promptfoo installs through npm, npx or Homebrew; Homebrew supports Mac and Linux. It requires Node.js 22.22.0 or newer and recommends Node.js 24 LTS, so teams must account for that runtime requirement. Community runs locally or self-hosted. Enterprise adds managed cloud deployment, while On-Premise runs on the customer’s infrastructure with complete data isolation and a dedicated runner.
The site displays SOC 2 and ISO 27001 certification badges. Community support is community-based; Enterprise adds professional services, priority support and SLA guarantees.
Pricing
| Plan | Price and terms | What it includes |
|---|---|---|
| Community | Free | 10k red-team probes/month, all LLM evaluation features, all model providers and integrations, local or self-hosted deployment, community support. |
| Enterprise | Custom pricing based on team size and needs | Custom red-teaming limits, team sharing, continuous monitoring, SSO, API access, managed cloud deployment, priority support and SLA. |
| On-Premise | Custom | All Enterprise features, deployment on the customer’s infrastructure, complete data isolation, dedicated runner and assigned deployment engineer. |
Community is unusually broad for a free starting point, retaining evaluation features, providers and integrations rather than reserving them for paid tiers. Its trade-off is the 10k monthly red-team probe limit and community support. Enterprise suits teams that need collaboration, monitoring, managed deployment or service commitments; On-Premise is the fit when infrastructure control and data isolation are priorities. Both paid plans use custom pricing.
Platforms
Promptfoo supports API, Linux, macOS, self-hosted, web and Windows. The range accommodates local and hosted workflows, though installation requires the specified Node.js version.
Who it's for
Promptfoo suits developers and organizations that need both model-output evaluation and AI red-team testing, particularly when they want to connect checks to CI/CD. It is less suitable for teams unwilling to manage its runtime requirement or whose probe volume exceeds Community’s cap and cannot justify custom-priced plans.
Pros and cons
- Pros: Community includes all evaluation features, providers and integrations, making it possible to start without a paid tier.
- Pros: Evaluation and red teaming cover both output comparison and security risks, including agent tool use.
- Pros: Local, self-hosted, managed-cloud and on-premise options support different deployment needs.
- Cons: Community red teaming is capped at 10k probes per month, which may constrain heavier testing.
- Cons: Node.js 22.22.0 or newer is required, adding a runtime dependency for installations.
- Cons: Enterprise and On-Premise pricing is custom, so teams cannot compare their cost from a fixed price.
Alternatives
For a focused open-source evaluation runner, DeepEval offers local and CI/CD testing under an Apache 2.0 license. Maxim AI is another freemium option; its free Developer plan allows up to 3 seats, 1 workspace and 10k logs per month, with 3-day retention and email support.
Giskard, Whisper, Arena (formerly Chatbot Arena), Braintrust, Galileo and Parea AI are other options to compare.
Explore LLM Evaluation Tools, AI Prompt Management Software, AI Red Teaming Tools, AI LLM Evaluation Tools, AI Security Testing Tools and AI Agent Evaluation Tools.
Verdict
Choose Promptfoo if you want a single workflow for evaluating prompts across providers and probing AI applications for security risks, with a free plan that retains broad evaluation capabilities. Look elsewhere if Node.js is an obstacle, the monthly probe cap is too restrictive, or you need a predictable paid price rather than custom terms.
The verdict
Promptfoo combines prompt evaluation with red-team testing and integrations for development workflows. The free Community plan includes broad evaluation features, while higher-tier deployment and support options have custom pricing.
Questions people ask
Is Promptfoo free?
Yes. The Community plan costs nothing and includes all LLM evaluation features, providers, integrations, and local or self-hosted running.
Which platforms does Promptfoo support?
It supports API, Linux, macOS, self-hosted, web, and Windows deployments.
What providers can it evaluate?
The getting-started guide lists more than 60 providers, including OpenAI, Anthropic, Google, and local models such as Ollama.
Does Promptfoo integrate with CI/CD tools?
Yes. Listed integrations include GitHub Actions, GitLab CI, Jenkins, Azure Pipelines, CircleCI, and Bitbucket Pipelines.
Is Promptfoo open source?
Promptfoo says it remains open source and is now part of OpenAI.
What the maker publishes
- Acquisition news
- Promptfoo announced on March 9, 2026 that it had agreed to be acquired by OpenAI while remaining open source. source
- Browser testing
- An optional Playwright-based browser provider supports Chromium, Edge, Firefox, and WebKit interactions. source
- CI/CD
- Promptfoo can run evaluations and security checks in CI/CD pipelines using configuration files and provider credentials. source
- Cloud viewer
- The browser viewer is free at promptfoo.app and supports private organization links plus self-hosted sharing endpoints. source
- Company
- Promptfoo says it is based in San Francisco, California, and is now part of OpenAI while remaining open source. source
- Company founding
- Promptfoo's co-founder said the company was founded in 2024. source
- Company legal name
- Crunchbase lists the legal name as Promptfoo, Inc. source
- Company size
- LinkedIn lists Promptfoo as a privately held software-development company with 11–50 employees. source
- Compliance testing
- Its GDPR preset tests privacy, access control, personal-data exposure, deletion flows, and automated-decision risks, but the documentation says it is not legal certification. source
- Custom extensibility
- Developers can write JavaScript, TypeScript, Python, Ruby, or script-based custom providers for APIs not built in. source
- Deployment
- The Community plan can run locally or be self-hosted, while Enterprise includes managed cloud deployment and On-Premise supports deployment on the customer’s infrastructure. source
- Documentation depth
- The documentation includes installation, evaluations, assertions, runtime, red teaming, providers, integrations, code scanning, model audit, enterprise, API references, and release notes. source
- Evaluation
- It tests prompts across multiple providers, grades outputs with assertions, and calculates accuracy, safety, and performance metrics. source
- Evaluations
- Promptfoo evaluates prompts across multiple models and opens a web view for comparing outputs. source
- Founded
- 2024 source
- Funding
- Crunchbase lists seed and Series A rounds, six investors including Insight Partners and Andreessen Horowitz, and 11–50 employees. source
- Headquarters
- Promptfoo identifies San Francisco, California as its base and headquarters. source
- HTTP API support
- The HTTP provider can call arbitrary endpoints, build requests from templates, transform responses, authenticate, and handle tool-calling formats. source
- Independent package evidence
- Homebrew lists promptfoo as MIT licensed and provides macOS Apple Silicon plus Linux ARM64 and x86_64 bottles. source
- Installation
- Promptfoo can be installed using npm, npx, or Homebrew, and the installation guide identifies Homebrew as supporting Mac and Linux. source
- Installation
- Installation is available through npm, npx, Homebrew, and a project dependency; npm and npx require Node.js 22.22.0 or newer. source
- Integrations
- Listed integrations include GitHub Actions, GitLab CI, Jenkins, Azure Pipelines, CircleCI, Bitbucket Pipelines, Splunk, Burp Suite, Google Sheets, and MCP. source
- Integrations
- Documented integrations include LangChain, LangGraph, CrewAI, MCP, Splunk, Burp Suite, GitHub Actions, GitLab, Jenkins, CircleCI, Azure Pipelines, and Bitbucket Pipelines. source
- License
- The open-source project is distributed under the MIT license. source
Checked against the maker's pages on 15 Oct 2026.