Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

Rhesis AI

rhesis.ai

  • Free plan
  • Linux
  • Web
  • API
  • Self-hosted
7.4 Club score Visit rhesis.ai

What Rhesis AI is

Rhesis AI is an open-source collaboration layer for building and evaluating AI agents. Domain experts, engineers, and product managers can share a workspace to review test cases and agent versions, add annotations, and record expectations. Teams can generate test sets from application requirements or constraints, including single-turn prompts and multi-turn conversations, then run them against an application endpoint and save metric-based results. Rhesis Tracing uses OpenTelemetry to capture model calls, tool use, and retrieval operations in testing or production. Evaluation options include offline checks, online trace metrics, LLM-as-a-judge, custom metrics, safety testing, and adversarial red-teaming. It connects to cloud and local model providers through LiteLLM and supports DeepEval and Ragas metrics, plus Garak probes. Its REST API manages test sets and runs, retrieves results, and inspects traces for integrations and CI/CD. Rhesis can run as a managed service, locally with Docker, or self-hosted; the site says it is developed in Germany and hosted in the EU. Free options include a self-hosted edition and Rhesis Cloud Free.

Who it suits

Rhesis is intended for domain experts, engineers, and product managers working on AI agents. It suits teams that need to create evaluations, inspect traces, or connect testing to CI/CD workflows.

Good to know, and what to watch

  • Shared workspace for tests and agent versions
  • Generates single-turn and multi-turn test sets
  • Captures calls, tools, and retrieval traces
  • Supports cloud and local model providers
  • Free cloud and self-hosted options
  • Cloud Free has a monthly allowance
  • Self-hosting requires local or hosted deployment setup

Rhesis Cloud Free

Free

Self-hosted open source edition

Free

Free plan

The verdict

Rhesis brings test creation, evaluation, and tracing into a shared workflow for AI-agent teams. Its free options and deployment choices offer flexibility, while teams should check whether the Cloud Free allowance meets their needs.

What the maker publishes

API and CI/CD
The REST API supports managing test sets, triggering test runs, fetching results, and inspecting traces for custom integrations and CI/CD pipelines. source
Deployment and data location
Rhesis can run as a managed service, locally with Docker, or self-hosted; the site says it is developed in Germany and hosted in the EU. source
Evaluation integrations
Rhesis supports DeepEval and Ragas metrics and can import Garak probes as test sets for adversarial scanning. source
Evaluations
Rhesis runs test sets against an application endpoint, evaluates responses using configured metrics, and records results in test runs. source
Headquarters
Potsdam, Germany source
Intended users
Rhesis describes itself as built for experts, engineers, and product managers working on AI agents. source
Model integrations
Rhesis supports cloud and local model providers through LiteLLM, including OpenAI, Anthropic, Google Gemini, Mistral, Cohere, Groq, Together AI, Ollama, and vLLM. source
Product
Rhesis is an open-source collaboration layer that brings domain-expert knowledge into AI agent development. source
Security testing
Rhesis integrates Garak to probe applications for issues such as prompt injection, data leakage, and harmful content generation. source
Support
The getting-started page points users to in-app support, documentation, a support email, GitHub, and Discord. source
Team workflow
Domain experts, engineers, and product managers can share a workspace for reviews, annotations, and expectations attached to test cases and agent versions. source
Test generation
Rhesis can generate test sets from application requirements or constraints, including single-turn prompts and multi-turn conversations. source
Tool integrations
The integrations page lists Notion, GitHub, Jira, GitLab, Asana, Azure DevOps, Linear, and Shortcut as tool connections. source
Tracing
Rhesis Tracing uses OpenTelemetry to capture LLM calls, tool invocations, and retrieval operations in test or production mode. source

Checked against the maker's pages on 30 Sept 2026.