AgentDojo
agentdojo.spylab.ai
- Free plan
- Linux
- API
- Self-hosted
6.9 Club score Visit agentdojo.spylab.ai
What the maker publishes
- Affiliations
- The listed authors are affiliated with ETH Zurich and Invariant Labs. source
- API status
- The package API is still under development and may change. source
- Benchmark
- The benchmark script runs selected models, defenses, attacks, task suites, and tasks. source
- Contribute results
- Users can submit benchmark results by opening a repository pull request that includes a brief description and an implementation. source
- Custom attacks
- Users can create and plug in custom attacks that target a pipeline, task suite, user task, and injection task. source
- Custom benchmarks
- Users can define task suites and tasks to extend the provided benchmarks or create a new benchmark from scratch. source
- Custom pipelines
- Pipeline components can be combined into custom agent pipelines, including defense components. source
- Defense research
- The results page reports model, defense, and attack combinations and says it is not a leaderboard because not all combinations were tested. source
- Function runtime
- Its functions runtime defines and runs Python functions for use with function-calling LLMs, including functions that operate on stateful environments. source
- Install
- The quickstart installs AgentDojo with `pip install agentdojo`. source
- License
- The project's GitHub page identifies its license as MIT. source
- Makers
- The project page lists authors affiliated with ETH Zurich and Invariant Labs. source
- Model integrations
- Documented tool calling LLM implementations include OpenAI GPT, Anthropic Claude, Google Gemini, Cohere Command, and a prompting interface for other models. source
- Model providers
- Documented tool-calling LLM implementations support OpenAI, Anthropic, Google, and Cohere; the docs also describe using Llama 3 70B through TogetherAI’s API. source
- Pipeline components
- Agent pipelines combine elements such as system messages, LLMs, tool execution, and defense components. source
- Prompt injection detector
- The prompt injection detector requires installing the `transformers` extra. source
- Purpose
- AgentDojo is a dynamic environment for evaluating prompt injection attacks and defenses for LLM agents. source
- Research use
- The project asks research users to consider citing its 2024 paper. source
- Results limitation
- The results page says its results are not a leaderboard because not all models were tested with all attacks and defenses. source
- Tool runtime
- Its functions runtime registers and runs Python functions for use with function calling LLMs, including functions that use stateful environments. source
- Trace inspection
- The results page says users can inspect run traces, including agent trajectories, attacks, and defenses, using Invariant Explorer. source
- Use case
- The documentation presents AgentDojo as a framework for researchers to evaluate attacks and defenses and build benchmarks. source
Checked against the maker's pages on 8 Oct 2026.