RAMPART
microsoft.github.io
- Free plan
- Windows
- macOS
- Linux
- API
What RAMPART is
RAMPART is a free, MIT-licensed, open-source framework for safety testing agentic AI applications. It runs as a pytest-native workflow: you connect an agent through an adapter, and RAMPART supplies execution strategies, evaluators, pytest integration, and reporting. Its attack tests treat a detected attack as UNSAFE, while probes treat detected expected behavior as SAFE. The documented built-in attack checks cross-prompt injection, including malicious content planted in data sources and retrieved by an agent. The built-in probe checks expected responses, tool use, or side effects, making it suitable for regression testing. Results can show SAFE, UNSAFE, UNDETERMINED, or ERROR, with trace and evaluation details. RAMPART registers automatically as a pytest plugin and provides harm and trial markers. It produces structured JSON reports, supports custom report sinks, and combines results from parallel pytest-xdist workers. Installation requires Python 3.11 or later and uses uv or pip on Linux, macOS, or Windows. PyRIT is installed for LLM interaction, prompt normalization, and conversation memory. An optional OneDrive extra supports placing XPIA payloads in OneDrive.
Who it suits
RAMPART suits developers writing safety and security tests for AI agents, especially teams using pytest and CI workflows. It is self-hosted and supports custom test cases, prompt-injection checks, data-leakage tests, and unsafe-output tests.
Good to know, and what to watch
- Free and MIT-licensed open source
- Runs natively within pytest workflows
- Built-in cross-prompt injection attack test
- Structured JSON and custom report sinks
- Parallel pytest-xdist execution with unified reports
- Requires Python 3.11 or later
- Mixed RAMPART versions across workers unsupported
- Fixture-teardown results excluded from streaming
- OneDrive integration requires optional extra
The verdict
RAMPART focuses on repeatable agent safety testing inside pytest. Its attack and probe model, JSON reporting, and parallel-run support fit teams that need test results in development or CI workflows.
What the maker publishes
- Attack and probe tests
- Its two test categories are attacks, where a detected attack means UNSAFE, and probes, where detected expected behavior means SAFE. source
- Attack testing
- Its XPIA tests check whether malicious content planted in data sources can manipulate an agent when retrieved. source
- Behavior probes
- Behavioral probes check for expected responses, tool use, or side effects and are described as useful for regression testing. source
- Evaluation output
- Results include a safety status of SAFE, UNSAFE, UNDETERMINED, or ERROR, along with trace and evaluation details. source
- Included tests
- The documented built-in attack is cross-prompt injection (XPIA), and the documented probe verifies expected agent behavior. source
- Install requirements
- The installation guide requires Python 3.11 or later and supports installation with uv or pip. source
- Integration model
- Users provide an adapter connecting their agent to RAMPART, which supplies execution strategies, evaluators, pytest integration, and reporting. source
- Intended users
- The project describes its framework as a developer-friendly way to write and run safety and security tests for AI agents. source
- License
- The GitHub repository identifies RAMPART as MIT-licensed. source
- Limitations
- The xdist documentation says results recorded only during fixture teardown are excluded from report streaming and mixed RAMPART versions across controller and workers are unsupported. source
- Maker
- The documentation identifies Microsoft Corporation as its copyright holder. source
- OneDrive
- An optional onedrive extra installs dependencies for using the built-in OneDriveSurface to place XPIA payloads in OneDrive. source
- Optional integration
- The optional onedrive extra adds dependencies for using the built-in OneDriveSurface to place XPIA payloads in OneDrive. source
- Parallel execution
- RAMPART supports pytest-xdist parallel runs and produces a unified report across worker processes. source
- Purpose
- RAMPART is a pytest-native safety testing framework for agentic AI applications that orchestrates attacks or probes, evaluates outcomes, and reports results. source
- Purpose
- RAMPART is a pytest-native safety testing framework for agentic AI applications. source
- PyRIT
- RAMPART uses PyRIT for LLM interaction, prompt normalization, and conversation memory, and installs PyRIT as a dependency. source
- PyRIT integration
- RAMPART uses PyRIT for LLM interaction, prompt normalization, and conversation memory, and installs PyRIT as a dependency. source
- pytest integration
- RAMPART registers as a pytest plugin automatically and provides harm and trial markers. source
- pytest plugin
- RAMPART registers automatically as a pytest plugin through the pytest11 entry point, without requiring conftest.py activation configuration. source
- Python requirement
- Installation requires Python 3.11 or later and uses uv or pip for package management. source
- Reporting
- RAMPART provides structured JSON reporting, including a built-in JSON file report sink and a hook for custom sinks. source
- Reporting and CI
- Built-in JSON report sinks and pytest-xdist support let teams collect reports for CI dashboards across parallel workers. source
- Security boundary
- For xdist worker serialization, RAMPART documents JSON-safe payloads, rejection of unknown schema versions, a default 16 MiB per-result cap, and ANSI escape stripping. source
Checked against the maker's pages on 9 Oct 2026.