Inspect AI
inspect.aisi.org.uk
- Free plan
- Browser extension
- API
What Inspect AI is
Inspect AI is a free, open-source framework for evaluating large language models across coding, agentic tasks, reasoning, knowledge, behavior, and multimodal understanding. Evaluations combine datasets, solvers, and scorers, and more than 200 pre-built evaluations are available. Inspect View provides web-based monitoring and visualization of results, while the VS Code extension supports evaluation-log browsing, task running and debugging, and option configuration. The framework includes built-in support for more than 20 model providers and local inference through Hugging Face, vLLM, and SGLang. Standard tools cover web search, code execution, text editing, computer use, and web browsing, with MCP tool support. Agents from OpenAI Agents SDK, Pydantic AI, LangChain, Claude Code, Codex CLI, and Gemini CLI can be integrated. Untrusted model code can run through Docker, Kubernetes, Modal, Proxmox, Vagrant, and other systems using an extension API. Inspect is self-hosted; installation uses PyPI, and development and runs require model access.
Who it suits
Inspect AI suits teams building or assessing language-model systems that need reusable evaluations, model integrations, agent support, and safety or human-review workflows. It is aimed at technical users comfortable with self-hosted, model-dependent tooling.
Good to know, and what to watch
- Open-source framework with a free plan
- More than 200 pre-built evaluations
- Supports over 20 model providers
- VS Code evaluation-log and debugging tools
- Multiple sandbox deployment options
- Self-hosted deployment required
- Evaluation runs require access to a model
- Installation uses pip
- Technical setup is required
Plans
Every plan and priceInspect AI
Open source
The verdict
Inspect AI provides a broad evaluation framework with reusable components, provider integrations, agent connections, and sandboxing. Its self-hosted setup and requirement for model access make it most suitable for technical evaluation teams.
What the maker publishes
- Agent integrations
- Inspect can integrate agents built with OpenAI Agents SDK, Pydantic AI, and LangChain, as well as CLI agents such as Claude Code, Codex CLI, and Gemini CLI. source
- Editor integration
- The Inspect VS Code extension provides tools for browsing evaluation logs, running and debugging tasks, and configuring options. source
- Evaluation results
- Inspect View is a web-based tool for monitoring and visualizing evaluations. source
- Founded
- 2024 source
- Installation requirement
- The getting-started instructions install Inspect from PyPI with pip install inspect-ai and say evaluation development and runs also require access to a model. source
- Model integrations
- Inspect has built-in support for more than 20 model providers and local inference with Hugging Face, vLLM, and SGLang. source
- Provider examples
- Documented provider groups include OpenAI, Anthropic, Google, AWS Bedrock, Azure AI, Hugging Face, vLLM, Ollama, and SGLang. source
- Purpose
- Inspect is an open-source framework for evaluating large language models across coding, agentic tasks, reasoning, knowledge, behavior, and multimodal understanding. source
- Reusable components
- Evaluations combine datasets, solvers, and scorers, and can use more than 200 pre-built evaluations. source
- Sandboxing
- The framework supports running untrusted model code in Docker, Kubernetes, Modal, Proxmox, Vagrant, and other systems through an extension API. source
- Tools and MCP
- Inspect includes standard tools for web search, code execution, text editing, computer use, and web browsing, and supports MCP tools. source
Checked against the maker's pages on 4 Oct 2026.