PyRIT
azure.github.io
- Free plan
- Windows
- macOS
- Linux
- Web
- API
- Self-hosted
What PyRIT is
PyRIT is a free, open-source framework for automated and human-led red teaming of generative AI systems. It supports single- and multi-turn strategies such as Crescendo, TAP, and Skeleton Key, while scenarios combine strategies and datasets for repeatable assessments of content harms, psychosocial risks, data leakage, and other objectives. Users can run assessments through a command-line scanner or interactive shell, use the CoPyRIT graphical interface, or build workflows with the framework. Documented targets include OpenAI, Azure, Anthropic, Google, Hugging Face, custom HTTP endpoints and WebSockets, and web apps tested with Playwright. Its components include targets, converters, scorers, memory, datasets, attacks, and scenarios. Converters can alter prompts through text operations or transform among text, images, audio, video, and files. Scorers can return Boolean or normalized 0.0–1.0 results using LLMs, Azure AI Content Safety, or custom logic. Built-in memory tracks conversations, scores, and attack results in SQLite or Azure SQL. PyRIT is self-hosted and requires a Python environment and configured AI endpoints; local installation lists Python 3.10 through 3.14 as prerequisites.
Who it suits
PyRIT suits teams assessing generative AI security and safety who can configure AI endpoints and run a self-hosted framework. It also offers interfaces for command-line use, interactive work, and graphical access.
Good to know, and what to watch
- Supports automated and human-led assessments.
- Includes repeatable scenarios for several risk areas.
- Works with named services, custom endpoints, and web apps.
- Offers multiple assessment interfaces and workflow options.
- Requires a Python environment and configured AI endpoints.
- Local installation lists Python 3.10 through 3.14.
- Frontend contributor setup requires Node.js 22 or higher.
Plans
Every plan and pricePyRIT
Free plan
Getting started
- Open the PyRIT website
- Install locally with pip or uv, or choose Docker
- Use Python 3.10 through 3.14 for local installation
- Configure the AI endpoints you plan to assess
- Run assessments with the scanner, shell, CoPyRIT or custom workflows
Limits to check
The free PyRIT plan is an open-source framework rather than a hosted allowance, and it requires a Python environment plus configured AI endpoints. Local installation supports Python 3.10 through 3.14; frontend contributor setup requires Node.js 22 or higher.
The Club's review
PyRIT is an open-source toolkit for automated and human-directed red-team assessments of generative AI. It is aimed at security practitioners and developers comfortable with Python, self-hosting and provider configuration. Its breadth and modularity reward teams building tailored evaluations, while those wanting a managed, ready-to-run service should consider alternatives.
Overview
PyRIT brings attack orchestration, datasets, prompt transformation, scoring and result storage into one framework. Assessments can be driven from a command-line scanner or interactive shell, explored through the CoPyRIT graphical interface, or assembled programmatically. This range suits both analysts running guided investigations and engineers integrating repeatable checks into their own workflows, but it also means the user owns environment setup and assessment design.
Documented targets cover OpenAI, Azure, Anthropic, Google and Hugging Face, plus custom HTTP endpoints, WebSockets and web applications exercised with Playwright. That provider and interface coverage is useful for organizations testing several models or application surfaces. PyRIT supports prompt-injection, jailbreak, data-leakage and unsafe-output testing, along with custom cases.
Key features
Strategies and repeatable scenarios
Single-turn and multi-turn approaches include Crescendo, TAP and Skeleton Key. Scenarios combine strategies with datasets for repeatable work on content harms, psychosocial risks, data exposure and other objectives. Teams can therefore compare runs against a defined set of probes instead of treating every exercise as an ad hoc conversation; the trade-off is that someone must select, tune and maintain those scenarios.
Modular transformations and scoring
The framework separates targets, converters, scorers, memory, datasets, attacks and scenarios. Converters can encode, obfuscate, translate or semantically alter prompts, and can move content among text, images, audio, video and files. Scorers return Boolean results or normalized values from 0.0 to 1.0, using LLMs, Azure AI Content Safety or custom logic. This is a strong foundation for organization-specific policies, although meaningful results depend on choosing suitable converters and scoring rules.
Results and interfaces
Built-in memory records conversations, scores and attack outcomes in SQLite or Azure SQL. The scanner and shell favor repeatable technical runs, while CoPyRIT offers a graphical route and the framework APIs support bespoke orchestration. In CoPyRIT, a key entered while creating a target remains in memory rather than being written to disk. For shared or deployed configurations, PyRIT recommends Azure Key Vault; plaintext .env files are less secure.
Pricing
| Plan | Price | What it means |
|---|---|---|
| PyRIT | Free | Open-source framework; requires a Python environment and configured AI endpoints. |
There is no license fee or subscription. The economic advantage is clearest for teams that already operate the required Python and model infrastructure; provider API usage and the work of configuring targets remain part of adoption.
Platforms
PyRIT supports API, Linux, macOS, self-hosted, web and Windows deployments. Local installation uses pip or uv, with separate Docker options. The local setup requires Python 3.10 through 3.14, and contributor work on the frontend requires Node.js 22 or later. These prerequisites make it suitable for controlled engineering environments, not users seeking a browser-only service.
Who it's for
Choose PyRIT when a security or AI engineering team needs provider-spanning red-team coverage, repeatable datasets and the freedom to define converters, scorers and workflows. It also fits organizations that need self-hosted handling of assessment data and can manage their own endpoint credentials. It is a poor match for teams without Python capability, those unwilling to maintain infrastructure, or buyers seeking packaged usage quotas and vendor-managed operations.
Pros and cons
Pros
- Broad target coverage: supports major model providers, custom endpoints, WebSockets and Playwright-tested web apps.
- Deep customization: modular attacks, converters, scorers and custom cases accommodate organization-specific tests.
- Repeatable assessments: scenarios pair strategies with datasets across several harm and leakage objectives.
- No software fee: the open-source plan costs nothing.
- Multiple working styles: scanner, shell, graphical interface and framework APIs serve different operator preferences.
Cons
- Operational responsibility: users must provide a Python environment, configure AI endpoints and maintain the self-hosted deployment.
- Prerequisite overhead: local installation spans Python 3.10–3.14, with Node.js 22+ for frontend contributors.
- Credential governance is yours: secure shared deployments require practices such as Azure Key Vault rather than plaintext environment files.
Alternatives
OpenSecureAI Scanner is a better fit when a freemium product with hosted remediation and a published Pro allowance of 50,000 API requests per month is preferable.
Promptfoo suits teams wanting a freemium Community plan with 10,000 red-team probes per month and local or self-hosted operation.
AIRTA Red Team offers another free, open-source route with a bring-your-own-LLM-key model.
Project Moonshot is an alternative free platform for teams comparing broader API, web and self-hosted availability.
Augustus is free, Apache 2.0-licensed open-source software for Linux, macOS, Windows and self-hosted use.
Basilisk is a free AGPL-3.0 option available through CLI, desktop, Docker or source distribution.
PromptGuard is worth considering for a freemium scanning service with a 20,000-scan monthly free tier and a $19 monthly Team plan.
F5 BIG-IP APM is a paid alternative with a free trial and perpetual-license options.
See the AI Security Testing Tools list for additional choices.
Verdict
PyRIT is the right choice for technically capable teams that want a free, self-hosted foundation for customized generative-AI red teaming across multiple providers and test objectives. Its main reason to choose it is the combination of modular attack, conversion and scoring components with repeatable scenarios. Look elsewhere if you need managed delivery, minimal setup or fixed hosted quotas rather than a framework you operate and configure yourself.
The verdict
PyRIT provides modular red-teaming workflows for testing generative AI systems and targets. Its self-hosted setup and environment prerequisites are important considerations before adoption.
Questions people ask
How much does PyRIT cost?
The PyRIT plan is free at no cost, with a free plan.
Which platforms does PyRIT support?
Listed platforms are API, Linux, macOS, self-hosted, web and Windows.
What AI targets can it assess?
Documented targets include OpenAI, Azure, Anthropic, Google, Hugging Face, custom HTTP endpoints, WebSockets and web apps tested with Playwright.
Can PyRIT test prompt injections and jailbreaks?
Yes. It supports prompt injection tests, jailbreak tests, data leakage tests, unsafe output tests and custom test cases.
How does PyRIT handle credentials?
In CoPyRIT, an API key entered while creating a target is held in memory and is not written to disk.
What the maker publishes
- Attack strategies
- It supports single-turn and multi-turn strategies including Crescendo, TAP, and Skeleton Key. source
- Compatibility limit
- The local installation page lists Python 3.10 through 3.14 as prerequisites, and the contributor setup page requires Node.js 22 or higher for the frontend. source
- Components
- The modular framework includes targets, converters, scorers, memory, datasets, attacks, and scenarios. source
- Credential handling
- In CoPyRIT, an API key entered when creating a target is stored in memory only and is not persisted to disk. source
- Installation
- The documentation provides local installation with pip or uv and separate Docker installation options. source
- Interfaces
- Users can run assessments with the command-line scanner and interactive shell, use the CoPyRIT graphical interface, or build workflows with the framework. source
- Memory
- Built-in memory can track conversations, scores, and attack results using SQLite or Azure SQL. source
- Prompt conversion
- Converters transform prompts through text operations such as encoding, obfuscation, translation, and semantic changes, as well as conversions among text, images, audio, video, and files. source
- Purpose
- PyRIT is an open-source framework for automated and human-led red teaming to assess the security and safety of generative AI systems. source
- Scenarios
- Its scenarios package attack strategies and datasets for repeatable assessments of content harms, psychosocial risks, data leakage, and other objectives. source
- Scoring
- Scorers can return true/false or normalized 0.0–1.0 scores and can use LLMs, Azure AI Content Safety, or custom logic. source
- Security
- PyRIT recommends Azure Key Vault for shared or deployed configuration and warns that plaintext .env files are less secure. source
- Targets
- Documented targets include OpenAI, Azure, Anthropic, Google, Hugging Face, custom HTTP endpoints and WebSockets, and web apps tested with Playwright. source
Checked against the maker's pages on 30 Sept 2026.