Rhesis AI

Rhesis AI is ranked #10 of 30 in AI LLM evaluation tools on The Geeks Club. It runs on API, Linux, Self-hosted, Web. There is a free plan.

Rhesis AI plans and pricing

All plans

Self-hosted open source edition Free Nothing metered. rhesis.ai · 30 Sept 2026

Rhesis Cloud Free Free Start free on Rhesis Cloud with a monthly allowance; exact allowance not stated on the opened pricing page. rhesis.ai · 30 Sept 2026

Compared on AI LLM evaluation tools

Free plan: Yes
Evaluation methods: offline evaluation, online trace metrics, LLM-as-a-judge, custom metrics, single-turn testing, multi-turn testing, adversarial red-teaming
Model support: OpenAI, Anthropic, Google Gemini, Azure OpenAI, Mistral, Cohere, Groq, Together AI, Perplexity, Replicate, Ollama, vLLM, LiteLLM Proxy
Safety evaluations: Yes
API access: Yes

Best Rhesis AI alternatives

See all 12

8.0 Weights & Biases Free plan

7.9 Evidently AI Free plan

7.7 Opik Free plan

7.6 Maxim AI Free plan

7.4 Vellum Free plan

7.3 Promptfoo Free plan

Rhesis AI

Product: Rhesis is an open-source collaboration layer that brings domain-expert knowledge into AI agent development.
Team workflow: Domain experts, engineers, and product managers can share a workspace for reviews, annotations, and expectations attached to test cases and agent versions.
Test generation: Rhesis can generate test sets from application requirements or constraints, including single-turn prompts and multi-turn conversations.
Evaluations: Rhesis runs test sets against an application endpoint, evaluates responses using configured metrics, and records results in test runs.
Tracing: Rhesis Tracing uses OpenTelemetry to capture LLM calls, tool invocations, and retrieval operations in test or production mode.
Model integrations: Rhesis supports cloud and local model providers through LiteLLM, including OpenAI, Anthropic, Google Gemini, Mistral, Cohere, Groq, Together AI, Ollama, and vLLM.
Evaluation integrations: Rhesis supports DeepEval and Ragas metrics and can import Garak probes as test sets for adversarial scanning.
Tool integrations: The integrations page lists Notion, GitHub, Jira, GitLab, Asana, Azure DevOps, Linear, and Shortcut as tool connections.
API and CI/CD: The REST API supports managing test sets, triggering test runs, fetching results, and inspecting traces for custom integrations and CI/CD pipelines.
Deployment and data location: Rhesis can run as a managed service, locally with Docker, or self-hosted; the site says it is developed in Germany and hosted in the EU.
Security testing: Rhesis integrates Garak to probe applications for issues such as prompt injection, data leakage, and harmful content generation.
Support: The getting-started page points users to in-app support, documentation, a support email, GitHub, and Discord.
Intended users: Rhesis describes itself as built for experts, engineers, and product managers working on AI agents.
Headquarters: Potsdam, Germany
Price from: Free

Visit site Alternatives

Where it ranks on The Geeks Club

Best AI LLM Evaluation Tools in 2026#10 of 30

Is Rhesis AI yours?

Claim it for free: prove the domain, then correct facts, plans and screenshots. An editor reviews every change.

Claim it free Promote Rhesis AI

Sources

rhesis.ai/about· checked 30 Sept 2026
docs.rhesis.ai/docs/getting-started/run-evaluations· checked 30 Sept 2026
docs.rhesis.ai/docs/test-runs/execution· checked 30 Sept 2026
docs.rhesis.ai/docs/tracing· checked 30 Sept 2026
docs.rhesis.ai/docs/integrations· checked 30 Sept 2026
docs.rhesis.ai/docs· checked 30 Sept 2026
docs.rhesis.ai/docs/getting-started· checked 30 Sept 2026
rhesis.ai· checked 28 Sept 2026
rhesis.ai/pricing· checked 30 Sept 2026