Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

RAMPART

microsoft.github.io

  • Free plan
  • Windows
  • macOS
  • Linux
  • API

What RAMPART is

RAMPART is a free, MIT-licensed, open-source framework for safety testing agentic AI applications. It runs as a pytest-native workflow: you connect an agent through an adapter, and RAMPART supplies execution strategies, evaluators, pytest integration, and reporting. Its attack tests treat a detected attack as UNSAFE, while probes treat detected expected behavior as SAFE. The documented built-in attack checks cross-prompt injection, including malicious content planted in data sources and retrieved by an agent. The built-in probe checks expected responses, tool use, or side effects, making it suitable for regression testing. Results can show SAFE, UNSAFE, UNDETERMINED, or ERROR, with trace and evaluation details. RAMPART registers automatically as a pytest plugin and provides harm and trial markers. It produces structured JSON reports, supports custom report sinks, and combines results from parallel pytest-xdist workers. Installation requires Python 3.11 or later and uses uv or pip on Linux, macOS, or Windows. PyRIT is installed for LLM interaction, prompt normalization, and conversation memory. An optional OneDrive extra supports placing XPIA payloads in OneDrive.

Who it suits

RAMPART suits developers writing safety and security tests for AI agents, especially teams using pytest and CI workflows. It is self-hosted and supports custom test cases, prompt-injection checks, data-leakage tests, and unsafe-output tests.

Good to know, and what to watch

  • Free and MIT-licensed open source
  • Runs natively within pytest workflows
  • Built-in cross-prompt injection attack test
  • Structured JSON and custom report sinks
  • Parallel pytest-xdist execution with unified reports
  • Requires Python 3.11 or later
  • Mixed RAMPART versions across workers unsupported
  • Fixture-teardown results excluded from streaming
  • OneDrive integration requires optional extra

The verdict

RAMPART focuses on repeatable agent safety testing inside pytest. Its attack and probe model, JSON reporting, and parallel-run support fit teams that need test results in development or CI workflows.

What the maker publishes

Attack and probe tests
Its two test categories are attacks, where a detected attack means UNSAFE, and probes, where detected expected behavior means SAFE. source
Attack testing
Its XPIA tests check whether malicious content planted in data sources can manipulate an agent when retrieved. source
Behavior probes
Behavioral probes check for expected responses, tool use, or side effects and are described as useful for regression testing. source
Evaluation output
Results include a safety status of SAFE, UNSAFE, UNDETERMINED, or ERROR, along with trace and evaluation details. source
Included tests
The documented built-in attack is cross-prompt injection (XPIA), and the documented probe verifies expected agent behavior. source
Install requirements
The installation guide requires Python 3.11 or later and supports installation with uv or pip. source
Integration model
Users provide an adapter connecting their agent to RAMPART, which supplies execution strategies, evaluators, pytest integration, and reporting. source
Intended users
The project describes its framework as a developer-friendly way to write and run safety and security tests for AI agents. source
License
The GitHub repository identifies RAMPART as MIT-licensed. source
Limitations
The xdist documentation says results recorded only during fixture teardown are excluded from report streaming and mixed RAMPART versions across controller and workers are unsupported. source
Maker
The documentation identifies Microsoft Corporation as its copyright holder. source
OneDrive
An optional onedrive extra installs dependencies for using the built-in OneDriveSurface to place XPIA payloads in OneDrive. source
Optional integration
The optional onedrive extra adds dependencies for using the built-in OneDriveSurface to place XPIA payloads in OneDrive. source
Parallel execution
RAMPART supports pytest-xdist parallel runs and produces a unified report across worker processes. source
Purpose
RAMPART is a pytest-native safety testing framework for agentic AI applications that orchestrates attacks or probes, evaluates outcomes, and reports results. source
Purpose
RAMPART is a pytest-native safety testing framework for agentic AI applications. source
PyRIT
RAMPART uses PyRIT for LLM interaction, prompt normalization, and conversation memory, and installs PyRIT as a dependency. source
PyRIT integration
RAMPART uses PyRIT for LLM interaction, prompt normalization, and conversation memory, and installs PyRIT as a dependency. source
pytest integration
RAMPART registers as a pytest plugin automatically and provides harm and trial markers. source
pytest plugin
RAMPART registers automatically as a pytest plugin through the pytest11 entry point, without requiring conftest.py activation configuration. source
Python requirement
Installation requires Python 3.11 or later and uses uv or pip for package management. source
Reporting
RAMPART provides structured JSON reporting, including a built-in JSON file report sink and a hook for custom sinks. source
Reporting and CI
Built-in JSON report sinks and pytest-xdist support let teams collect reports for CI dashboards across parallel workers. source
Security boundary
For xdist worker serialization, RAMPART documents JSON-safe payloads, rejection of unknown schema versions, a default 16 MiB per-result cap, and ANSI escape stripping. source

Checked against the maker's pages on 9 Oct 2026.