Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

AgentDojo

agentdojo.spylab.ai

  • Free plan
  • Linux
  • API
  • Self-hosted

What the maker publishes

Affiliations
The listed authors are affiliated with ETH Zurich and Invariant Labs. source
API status
The package API is still under development and may change. source
Benchmark
The benchmark script runs selected models, defenses, attacks, task suites, and tasks. source
Contribute results
Users can submit benchmark results by opening a repository pull request that includes a brief description and an implementation. source
Custom attacks
Users can create and plug in custom attacks that target a pipeline, task suite, user task, and injection task. source
Custom benchmarks
Users can define task suites and tasks to extend the provided benchmarks or create a new benchmark from scratch. source
Custom pipelines
Pipeline components can be combined into custom agent pipelines, including defense components. source
Defense research
The results page reports model, defense, and attack combinations and says it is not a leaderboard because not all combinations were tested. source
Function runtime
Its functions runtime defines and runs Python functions for use with function-calling LLMs, including functions that operate on stateful environments. source
Install
The quickstart installs AgentDojo with `pip install agentdojo`. source
License
The project's GitHub page identifies its license as MIT. source
Makers
The project page lists authors affiliated with ETH Zurich and Invariant Labs. source
Model integrations
Documented tool calling LLM implementations include OpenAI GPT, Anthropic Claude, Google Gemini, Cohere Command, and a prompting interface for other models. source
Model providers
Documented tool-calling LLM implementations support OpenAI, Anthropic, Google, and Cohere; the docs also describe using Llama 3 70B through TogetherAI’s API. source
Pipeline components
Agent pipelines combine elements such as system messages, LLMs, tool execution, and defense components. source
Prompt injection detector
The prompt injection detector requires installing the `transformers` extra. source
Purpose
AgentDojo is a dynamic environment for evaluating prompt injection attacks and defenses for LLM agents. source
Research use
The project asks research users to consider citing its 2024 paper. source
Results limitation
The results page says its results are not a leaderboard because not all models were tested with all attacks and defenses. source
Tool runtime
Its functions runtime registers and runs Python functions for use with function calling LLMs, including functions that use stateful environments. source
Trace inspection
The results page says users can inspect run traces, including agent trajectories, attacks, and defenses, using Invariant Explorer. source
Use case
The documentation presents AgentDojo as a framework for researchers to evaluate attacks and defenses and build benchmarks. source

Checked against the maker's pages on 8 Oct 2026.