Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

PyRIT

azure.github.io

  • Free plan
  • Windows
  • macOS
  • Linux
  • Web
  • API
  • Self-hosted
7.6 Club score Visit azure.github.io

What PyRIT is

PyRIT is a free, open-source framework for automated and human-led red teaming of generative AI systems. It supports single- and multi-turn strategies such as Crescendo, TAP, and Skeleton Key, while scenarios combine strategies and datasets for repeatable assessments of content harms, psychosocial risks, data leakage, and other objectives. Users can run assessments through a command-line scanner or interactive shell, use the CoPyRIT graphical interface, or build workflows with the framework. Documented targets include OpenAI, Azure, Anthropic, Google, Hugging Face, custom HTTP endpoints and WebSockets, and web apps tested with Playwright. Its components include targets, converters, scorers, memory, datasets, attacks, and scenarios. Converters can alter prompts through text operations or transform among text, images, audio, video, and files. Scorers can return Boolean or normalized 0.0–1.0 results using LLMs, Azure AI Content Safety, or custom logic. Built-in memory tracks conversations, scores, and attack results in SQLite or Azure SQL. PyRIT is self-hosted and requires a Python environment and configured AI endpoints; local installation lists Python 3.10 through 3.14 as prerequisites.

Who it suits

PyRIT suits teams assessing generative AI security and safety who can configure AI endpoints and run a self-hosted framework. It also offers interfaces for command-line use, interactive work, and graphical access.

Good to know, and what to watch

  • Supports automated and human-led assessments.
  • Includes repeatable scenarios for several risk areas.
  • Works with named services, custom endpoints, and web apps.
  • Offers multiple assessment interfaces and workflow options.
  • Requires a Python environment and configured AI endpoints.
  • Local installation lists Python 3.10 through 3.14.
  • Frontend contributor setup requires Node.js 22 or higher.

PyRIT

Free

Free plan

Getting started

  1. Open the PyRIT website
  2. Install locally with pip or uv, or choose Docker
  3. Use Python 3.10 through 3.14 for local installation
  4. Configure the AI endpoints you plan to assess
  5. Run assessments with the scanner, shell, CoPyRIT or custom workflows

Limits to check

The free PyRIT plan is an open-source framework rather than a hosted allowance, and it requires a Python environment plus configured AI endpoints. Local installation supports Python 3.10 through 3.14; frontend contributor setup requires Node.js 22 or higher.

The Club's review

PyRIT is an open-source toolkit for automated and human-directed red-team assessments of generative AI. It is aimed at security practitioners and developers comfortable with Python, self-hosting and provider configuration. Its breadth and modularity reward teams building tailored evaluations, while those wanting a managed, ready-to-run service should consider alternatives.

Overview

PyRIT brings attack orchestration, datasets, prompt transformation, scoring and result storage into one framework. Assessments can be driven from a command-line scanner or interactive shell, explored through the CoPyRIT graphical interface, or assembled programmatically. This range suits both analysts running guided investigations and engineers integrating repeatable checks into their own workflows, but it also means the user owns environment setup and assessment design.

Documented targets cover OpenAI, Azure, Anthropic, Google and Hugging Face, plus custom HTTP endpoints, WebSockets and web applications exercised with Playwright. That provider and interface coverage is useful for organizations testing several models or application surfaces. PyRIT supports prompt-injection, jailbreak, data-leakage and unsafe-output testing, along with custom cases.

Key features

Strategies and repeatable scenarios

Single-turn and multi-turn approaches include Crescendo, TAP and Skeleton Key. Scenarios combine strategies with datasets for repeatable work on content harms, psychosocial risks, data exposure and other objectives. Teams can therefore compare runs against a defined set of probes instead of treating every exercise as an ad hoc conversation; the trade-off is that someone must select, tune and maintain those scenarios.

Modular transformations and scoring

The framework separates targets, converters, scorers, memory, datasets, attacks and scenarios. Converters can encode, obfuscate, translate or semantically alter prompts, and can move content among text, images, audio, video and files. Scorers return Boolean results or normalized values from 0.0 to 1.0, using LLMs, Azure AI Content Safety or custom logic. This is a strong foundation for organization-specific policies, although meaningful results depend on choosing suitable converters and scoring rules.

Results and interfaces

Built-in memory records conversations, scores and attack outcomes in SQLite or Azure SQL. The scanner and shell favor repeatable technical runs, while CoPyRIT offers a graphical route and the framework APIs support bespoke orchestration. In CoPyRIT, a key entered while creating a target remains in memory rather than being written to disk. For shared or deployed configurations, PyRIT recommends Azure Key Vault; plaintext .env files are less secure.

Pricing

PlanPriceWhat it means
PyRITFreeOpen-source framework; requires a Python environment and configured AI endpoints.

There is no license fee or subscription. The economic advantage is clearest for teams that already operate the required Python and model infrastructure; provider API usage and the work of configuring targets remain part of adoption.

Platforms

PyRIT supports API, Linux, macOS, self-hosted, web and Windows deployments. Local installation uses pip or uv, with separate Docker options. The local setup requires Python 3.10 through 3.14, and contributor work on the frontend requires Node.js 22 or later. These prerequisites make it suitable for controlled engineering environments, not users seeking a browser-only service.

Who it's for

Choose PyRIT when a security or AI engineering team needs provider-spanning red-team coverage, repeatable datasets and the freedom to define converters, scorers and workflows. It also fits organizations that need self-hosted handling of assessment data and can manage their own endpoint credentials. It is a poor match for teams without Python capability, those unwilling to maintain infrastructure, or buyers seeking packaged usage quotas and vendor-managed operations.

Pros and cons

Pros

  • Broad target coverage: supports major model providers, custom endpoints, WebSockets and Playwright-tested web apps.
  • Deep customization: modular attacks, converters, scorers and custom cases accommodate organization-specific tests.
  • Repeatable assessments: scenarios pair strategies with datasets across several harm and leakage objectives.
  • No software fee: the open-source plan costs nothing.
  • Multiple working styles: scanner, shell, graphical interface and framework APIs serve different operator preferences.

Cons

  • Operational responsibility: users must provide a Python environment, configure AI endpoints and maintain the self-hosted deployment.
  • Prerequisite overhead: local installation spans Python 3.10–3.14, with Node.js 22+ for frontend contributors.
  • Credential governance is yours: secure shared deployments require practices such as Azure Key Vault rather than plaintext environment files.

Alternatives

OpenSecureAI Scanner is a better fit when a freemium product with hosted remediation and a published Pro allowance of 50,000 API requests per month is preferable.

Promptfoo suits teams wanting a freemium Community plan with 10,000 red-team probes per month and local or self-hosted operation.

AIRTA Red Team offers another free, open-source route with a bring-your-own-LLM-key model.

Project Moonshot is an alternative free platform for teams comparing broader API, web and self-hosted availability.

Augustus is free, Apache 2.0-licensed open-source software for Linux, macOS, Windows and self-hosted use.

Basilisk is a free AGPL-3.0 option available through CLI, desktop, Docker or source distribution.

PromptGuard is worth considering for a freemium scanning service with a 20,000-scan monthly free tier and a $19 monthly Team plan.

F5 BIG-IP APM is a paid alternative with a free trial and perpetual-license options.

See the AI Security Testing Tools list for additional choices.

Verdict

PyRIT is the right choice for technically capable teams that want a free, self-hosted foundation for customized generative-AI red teaming across multiple providers and test objectives. Its main reason to choose it is the combination of modular attack, conversion and scoring components with repeatable scenarios. Look elsewhere if you need managed delivery, minimal setup or fixed hosted quotas rather than a framework you operate and configure yourself.

The verdict

PyRIT provides modular red-teaming workflows for testing generative AI systems and targets. Its self-hosted setup and environment prerequisites are important considerations before adoption.

Questions people ask

How much does PyRIT cost?

The PyRIT plan is free at no cost, with a free plan.

Which platforms does PyRIT support?

Listed platforms are API, Linux, macOS, self-hosted, web and Windows.

What AI targets can it assess?

Documented targets include OpenAI, Azure, Anthropic, Google, Hugging Face, custom HTTP endpoints, WebSockets and web apps tested with Playwright.

Can PyRIT test prompt injections and jailbreaks?

Yes. It supports prompt injection tests, jailbreak tests, data leakage tests, unsafe output tests and custom test cases.

How does PyRIT handle credentials?

In CoPyRIT, an API key entered while creating a target is held in memory and is not written to disk.

What the maker publishes

Attack strategies
It supports single-turn and multi-turn strategies including Crescendo, TAP, and Skeleton Key. source
Compatibility limit
The local installation page lists Python 3.10 through 3.14 as prerequisites, and the contributor setup page requires Node.js 22 or higher for the frontend. source
Components
The modular framework includes targets, converters, scorers, memory, datasets, attacks, and scenarios. source
Credential handling
In CoPyRIT, an API key entered when creating a target is stored in memory only and is not persisted to disk. source
Installation
The documentation provides local installation with pip or uv and separate Docker installation options. source
Interfaces
Users can run assessments with the command-line scanner and interactive shell, use the CoPyRIT graphical interface, or build workflows with the framework. source
Memory
Built-in memory can track conversations, scores, and attack results using SQLite or Azure SQL. source
Prompt conversion
Converters transform prompts through text operations such as encoding, obfuscation, translation, and semantic changes, as well as conversions among text, images, audio, video, and files. source
Purpose
PyRIT is an open-source framework for automated and human-led red teaming to assess the security and safety of generative AI systems. source
Scenarios
Its scenarios package attack strategies and datasets for repeatable assessments of content harms, psychosocial risks, data leakage, and other objectives. source
Scoring
Scorers can return true/false or normalized 0.0–1.0 scores and can use LLMs, Azure AI Content Safety, or custom logic. source
Security
PyRIT recommends Azure Key Vault for shared or deployed configuration and warns that plaintext .env files are less secure. source
Targets
Documented targets include OpenAI, Azure, Anthropic, Google, Hugging Face, custom HTTP endpoints and WebSockets, and web apps tested with Playwright. source

Checked against the maker's pages on 30 Sept 2026.