Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

Promptfoo

promptfoo.dev

  • Free plan
  • Open source
  • Windows
  • macOS
  • Linux
  • Web
  • API
  • Self-hosted
8.5 Club score Visit promptfoo.dev

What Promptfoo is

Promptfoo is a platform for building and evaluating AI applications. Developers can compare prompt results across models in a web view, then use red-team simulations to look for risks such as prompt injection, jailbreaks, data leaks, business-rule violations, and unsafe agent tool use. Its getting-started guide lists 60+ providers, including OpenAI, Anthropic, Google, and local models such as Ollama. Integrations include CI systems, security tools, Google Sheets, and MCP. Promptfoo supports custom metrics, LLM-as-a-judge evaluations, safety checks, human review workflows, and CI/CD integration. The Community plan costs nothing and includes all LLM evaluation features, all model providers and integrations, and 10k red-team probes per month; it can run locally or be self-hosted. Enterprise and On-Premise plans have custom pricing, with different deployment options. Installation is available through npm, npx, or Homebrew. It requires Node.js 22.22.0 or newer and recommends Node.js 24 LTS. The company says it remains open source and is now part of OpenAI.

Who it suits

Promptfoo suits developers and enterprise teams evaluating prompts and checking AI applications for security risks. Its local, self-hosted, managed-cloud, and on-premise options serve teams with different deployment needs.

Good to know, and what to watch

  • Compares prompt outputs across multiple models.
  • Red-team simulations probe several AI security risks.
  • Supports 60+ providers and listed integrations.
  • Community includes 10k red-team probes per month.
  • Offers local, self-hosted, cloud, and on-premise deployment.
  • Requires Node.js 22.22.0 or newer.
  • Community red-team use is limited to 10k probes monthly.
  • Enterprise and On-Premise pricing is custom and unlisted.

Enterprise

—

On-Premise

—

Community

Free

Free plan

Getting started

  1. Open the Promptfoo website
  2. Install Promptfoo with npm, npx, or Homebrew
  3. Use Node.js 22.22.0 or newer; Node.js 24 LTS is recommended
  4. Choose Community for local or self-hosted use, or select Enterprise or On-Premise
  5. Connect your model providers and listed integrations

Limits to check

Community includes 10k red-team probes per month. Enterprise and On-Premise use custom red-teaming limits.

The Club's review

Promptfoo is an evaluation and security-testing platform for developers building AI applications, from solo practitioners to enterprise teams. It is strongest when teams want to compare model outputs and probe for risks in the same development workflow; the free plan’s 10k monthly probe cap and Node.js requirement are meaningful constraints.

Overview

Promptfoo combines prompt evaluation across models with simulated red-team testing. Evaluations open a web view for comparing outputs; red teaming targets risks including prompt injection, jailbreaks, data leaks, business-rule violations and insecure agent tool use. The company was founded in 2024, is based in San Francisco, and says it is now part of OpenAI while remaining open source.

Key features

Evaluation and provider coverage

Promptfoo supports custom metrics, LLM-as-a-judge, safety evaluations and human review workflows. Its getting-started guide names more than 60 providers, including OpenAI, Anthropic, Google and local models such as Ollama. That breadth is useful for teams comparing models across hosted and local environments, rather than evaluating against just one provider.

Red teaming and integrations

Red-team runs simulate users to look for weaknesses in prompts and agents. Community includes 10k probes per month, with each probe defined as one request to the target system. That offers a substantial starting allowance, but teams running frequent or large-scale tests will need to monitor usage or consider custom limits.

Integrations include GitHub Actions, GitLab CI, Jenkins, Azure Pipelines, CircleCI and Bitbucket Pipelines, alongside Splunk, Burp Suite, Google Sheets and MCP. CI/CD integration can put evaluation into development workflows, while the security and data tools extend connections beyond deployment pipelines.

Installation and deployment

Promptfoo installs through npm, npx or Homebrew; Homebrew supports Mac and Linux. It requires Node.js 22.22.0 or newer and recommends Node.js 24 LTS, so teams must account for that runtime requirement. Community runs locally or self-hosted. Enterprise adds managed cloud deployment, while On-Premise runs on the customer’s infrastructure with complete data isolation and a dedicated runner.

The site displays SOC 2 and ISO 27001 certification badges. Community support is community-based; Enterprise adds professional services, priority support and SLA guarantees.

Pricing

PlanPrice and termsWhat it includes
CommunityFree10k red-team probes/month, all LLM evaluation features, all model providers and integrations, local or self-hosted deployment, community support.
EnterpriseCustom pricing based on team size and needsCustom red-teaming limits, team sharing, continuous monitoring, SSO, API access, managed cloud deployment, priority support and SLA.
On-PremiseCustomAll Enterprise features, deployment on the customer’s infrastructure, complete data isolation, dedicated runner and assigned deployment engineer.

Community is unusually broad for a free starting point, retaining evaluation features, providers and integrations rather than reserving them for paid tiers. Its trade-off is the 10k monthly red-team probe limit and community support. Enterprise suits teams that need collaboration, monitoring, managed deployment or service commitments; On-Premise is the fit when infrastructure control and data isolation are priorities. Both paid plans use custom pricing.

Platforms

Promptfoo supports API, Linux, macOS, self-hosted, web and Windows. The range accommodates local and hosted workflows, though installation requires the specified Node.js version.

Who it's for

Promptfoo suits developers and organizations that need both model-output evaluation and AI red-team testing, particularly when they want to connect checks to CI/CD. It is less suitable for teams unwilling to manage its runtime requirement or whose probe volume exceeds Community’s cap and cannot justify custom-priced plans.

Pros and cons

  • Pros: Community includes all evaluation features, providers and integrations, making it possible to start without a paid tier.
  • Pros: Evaluation and red teaming cover both output comparison and security risks, including agent tool use.
  • Pros: Local, self-hosted, managed-cloud and on-premise options support different deployment needs.
  • Cons: Community red teaming is capped at 10k probes per month, which may constrain heavier testing.
  • Cons: Node.js 22.22.0 or newer is required, adding a runtime dependency for installations.
  • Cons: Enterprise and On-Premise pricing is custom, so teams cannot compare their cost from a fixed price.

Alternatives

For a focused open-source evaluation runner, DeepEval offers local and CI/CD testing under an Apache 2.0 license. Maxim AI is another freemium option; its free Developer plan allows up to 3 seats, 1 workspace and 10k logs per month, with 3-day retention and email support.

Giskard, Whisper, Arena (formerly Chatbot Arena), Braintrust, Galileo and Parea AI are other options to compare.

Explore LLM Evaluation Tools, AI Prompt Management Software, AI Red Teaming Tools, AI LLM Evaluation Tools, AI Security Testing Tools and AI Agent Evaluation Tools.

Verdict

Choose Promptfoo if you want a single workflow for evaluating prompts across providers and probing AI applications for security risks, with a free plan that retains broad evaluation capabilities. Look elsewhere if Node.js is an obstacle, the monthly probe cap is too restrictive, or you need a predictable paid price rather than custom terms.

The verdict

Promptfoo combines prompt evaluation with red-team testing and integrations for development workflows. The free Community plan includes broad evaluation features, while higher-tier deployment and support options have custom pricing.

Questions people ask

Is Promptfoo free?

Yes. The Community plan costs nothing and includes all LLM evaluation features, providers, integrations, and local or self-hosted running.

Which platforms does Promptfoo support?

It supports API, Linux, macOS, self-hosted, web, and Windows deployments.

What providers can it evaluate?

The getting-started guide lists more than 60 providers, including OpenAI, Anthropic, Google, and local models such as Ollama.

Does Promptfoo integrate with CI/CD tools?

Yes. Listed integrations include GitHub Actions, GitLab CI, Jenkins, Azure Pipelines, CircleCI, and Bitbucket Pipelines.

Is Promptfoo open source?

Promptfoo says it remains open source and is now part of OpenAI.

What the maker publishes

Acquisition news
Promptfoo announced on March 9, 2026 that it had agreed to be acquired by OpenAI while remaining open source. source
Browser testing
An optional Playwright-based browser provider supports Chromium, Edge, Firefox, and WebKit interactions. source
CI/CD
Promptfoo can run evaluations and security checks in CI/CD pipelines using configuration files and provider credentials. source
Cloud viewer
The browser viewer is free at promptfoo.app and supports private organization links plus self-hosted sharing endpoints. source
Company
Promptfoo says it is based in San Francisco, California, and is now part of OpenAI while remaining open source. source
Company founding
Promptfoo's co-founder said the company was founded in 2024. source
Company legal name
Crunchbase lists the legal name as Promptfoo, Inc. source
Company size
LinkedIn lists Promptfoo as a privately held software-development company with 11–50 employees. source
Compliance testing
Its GDPR preset tests privacy, access control, personal-data exposure, deletion flows, and automated-decision risks, but the documentation says it is not legal certification. source
Custom extensibility
Developers can write JavaScript, TypeScript, Python, Ruby, or script-based custom providers for APIs not built in. source
Deployment
The Community plan can run locally or be self-hosted, while Enterprise includes managed cloud deployment and On-Premise supports deployment on the customer’s infrastructure. source
Documentation depth
The documentation includes installation, evaluations, assertions, runtime, red teaming, providers, integrations, code scanning, model audit, enterprise, API references, and release notes. source
Evaluation
It tests prompts across multiple providers, grades outputs with assertions, and calculates accuracy, safety, and performance metrics. source
Evaluations
Promptfoo evaluates prompts across multiple models and opens a web view for comparing outputs. source
Founded
2024 source
Funding
Crunchbase lists seed and Series A rounds, six investors including Insight Partners and Andreessen Horowitz, and 11–50 employees. source
Headquarters
Promptfoo identifies San Francisco, California as its base and headquarters. source
HTTP API support
The HTTP provider can call arbitrary endpoints, build requests from templates, transform responses, authenticate, and handle tool-calling formats. source
Independent package evidence
Homebrew lists promptfoo as MIT licensed and provides macOS Apple Silicon plus Linux ARM64 and x86_64 bottles. source
Installation
Promptfoo can be installed using npm, npx, or Homebrew, and the installation guide identifies Homebrew as supporting Mac and Linux. source
Installation
Installation is available through npm, npx, Homebrew, and a project dependency; npm and npx require Node.js 22.22.0 or newer. source
Integrations
Listed integrations include GitHub Actions, GitLab CI, Jenkins, Azure Pipelines, CircleCI, Bitbucket Pipelines, Splunk, Burp Suite, Google Sheets, and MCP. source
Integrations
Documented integrations include LangChain, LangGraph, CrewAI, MCP, Splunk, Burp Suite, GitHub Actions, GitLab, Jenkins, CircleCI, Azure Pipelines, and Bitbucket Pipelines. source
License
The open-source project is distributed under the MIT license. source

Checked against the maker's pages on 15 Oct 2026.