Rhesis AI
rhesis.ai
- Free plan
- Linux
- Web
- API
- Self-hosted
What Rhesis AI is
Rhesis AI is an open-source collaboration layer for building and evaluating AI agents. Domain experts, engineers, and product managers can share a workspace to review test cases and agent versions, add annotations, and record expectations. Teams can generate test sets from application requirements or constraints, including single-turn prompts and multi-turn conversations, then run them against an application endpoint and save metric-based results. Rhesis Tracing uses OpenTelemetry to capture model calls, tool use, and retrieval operations in testing or production. Evaluation options include offline checks, online trace metrics, LLM-as-a-judge, custom metrics, safety testing, and adversarial red-teaming. It connects to cloud and local model providers through LiteLLM and supports DeepEval and Ragas metrics, plus Garak probes. Its REST API manages test sets and runs, retrieves results, and inspects traces for integrations and CI/CD. Rhesis can run as a managed service, locally with Docker, or self-hosted; the site says it is developed in Germany and hosted in the EU. Free options include a self-hosted edition and Rhesis Cloud Free.
Who it suits
Rhesis is intended for domain experts, engineers, and product managers working on AI agents. It suits teams that need to create evaluations, inspect traces, or connect testing to CI/CD workflows.
Good to know, and what to watch
- Shared workspace for tests and agent versions
- Generates single-turn and multi-turn test sets
- Captures calls, tools, and retrieval traces
- Supports cloud and local model providers
- Free cloud and self-hosted options
- Cloud Free has a monthly allowance
- Self-hosting requires local or hosted deployment setup
Plans
Every plan and priceRhesis Cloud Free
Self-hosted open source edition
Free plan
The verdict
Rhesis brings test creation, evaluation, and tracing into a shared workflow for AI-agent teams. Its free options and deployment choices offer flexibility, while teams should check whether the Cloud Free allowance meets their needs.
What the maker publishes
- API and CI/CD
- The REST API supports managing test sets, triggering test runs, fetching results, and inspecting traces for custom integrations and CI/CD pipelines. source
- Deployment and data location
- Rhesis can run as a managed service, locally with Docker, or self-hosted; the site says it is developed in Germany and hosted in the EU. source
- Evaluation integrations
- Rhesis supports DeepEval and Ragas metrics and can import Garak probes as test sets for adversarial scanning. source
- Evaluations
- Rhesis runs test sets against an application endpoint, evaluates responses using configured metrics, and records results in test runs. source
- Headquarters
- Potsdam, Germany source
- Intended users
- Rhesis describes itself as built for experts, engineers, and product managers working on AI agents. source
- Model integrations
- Rhesis supports cloud and local model providers through LiteLLM, including OpenAI, Anthropic, Google Gemini, Mistral, Cohere, Groq, Together AI, Ollama, and vLLM. source
- Product
- Rhesis is an open-source collaboration layer that brings domain-expert knowledge into AI agent development. source
- Security testing
- Rhesis integrates Garak to probe applications for issues such as prompt injection, data leakage, and harmful content generation. source
- Support
- The getting-started page points users to in-app support, documentation, a support email, GitHub, and Discord. source
- Team workflow
- Domain experts, engineers, and product managers can share a workspace for reviews, annotations, and expectations attached to test cases and agent versions. source
- Test generation
- Rhesis can generate test sets from application requirements or constraints, including single-turn prompts and multi-turn conversations. source
- Tool integrations
- The integrations page lists Notion, GitHub, Jira, GitLab, Asana, Azure DevOps, Linear, and Shortcut as tool connections. source
- Tracing
- Rhesis Tracing uses OpenTelemetry to capture LLM calls, tool invocations, and retrieval operations in test or production mode. source
Checked against the maker's pages on 30 Sept 2026.