DeepEval

DeepEval is ranked #7 of 30 in AI LLM evaluation tools on The Geeks Club. It runs on Linux, macOS, Self-hosted, Windows. There is a free plan.

DeepEval plans and pricing

All plans

DeepEval Free Open-source LLM evaluation framework · Apache 2.0 licensed · local and CI/CD test runner deepeval.com · 28 Sept 2026

Compared on AI LLM evaluation tools

Free plan: Yes
Evaluation methods: model
Tool-call checks: Yes
Trace ingestion: Yes
Safety evaluations: Yes
Regression runs: Yes
SDK language support: both

Best DeepEval alternatives

See all 12

8.0 Weights & Biases Free plan

7.9 Evidently AI Free plan

7.7 Opik Free plan

7.6 Maxim AI Free plan

7.4 Vellum Free plan

7.3 Promptfoo Free plan

DeepEval

Purpose: DeepEval is an open-source LLM evaluation framework for building evaluation pipelines to test AI systems.
Testing: It provides Pytest-native evaluations that run in CI/CD or as Python scripts.
Metrics: The site lists 50+ research-backed metrics, including hallucination, faithfulness, answer relevancy, summarization, toxicity, and bias.
Modalities: The framework supports evaluation of text, images, and audio, including conversational and voice evaluations.
Synthetic data: DeepEval can generate synthetic goldens from a knowledge base and simulate conversations across user personas.
Tracing: DeepEval traces agent steps so they can be graded and inspected in the terminal and test runner.
Integrations: Listed integrations include LangChain, Pydantic AI, OpenAI Agents, LangGraph, AWS AgentCore, Strands, Google ADK, LlamaIndex, and CrewAI.
Model providers: Evaluation model integrations include OpenAI, Azure OpenAI, Ollama, OpenRouter, Anthropic, Amazon Bedrock, Gemini, DeepSeek, Vertex AI, Grok, Moonshot, Portkey, vLLM, LM Studio, and LiteLLM.
Local telemetry: By default, DeepEval sends basic telemetry to PostHog, excludes personally identifiable information and stored results, and supports opting out with DEEPEVAL_TELEMETRY_OPT_OUT=1.
Cloud data: The maker says data sent to Confident AI is stored in databases in its private AWS cloud, except for organizations on the VIP plan.
Enterprise security: The enterprise page lists SSO, role-based access control, granular permissions, audit logs, SOC 2 Type II, GDPR compliance, and custom data retention.
Enterprise deployment: The enterprise offering is available on Confident AI Evals and can be self-hosted on a customer's infrastructure or run in the maker's cloud.
Support and collaboration: The enterprise page invites prospective customers to book a demo and describes shared workspaces, no-code evaluation workflows, and annotation queues.
Notable limit: The maker describes DeepEval OS as limited to pre-production testing, with results in local files and an engineer-owned test runner.
Price from: Free

Visit site Alternatives

Where it ranks on The Geeks Club

Best AI LLM Evaluation Tools in 2026#7 of 30
Best LLM Evaluation Tools in 2026#3 of 29
Best AI Agent Evaluation Tools in 2026#7 of 29

Is DeepEval yours?

Claim it for free: prove the domain, then correct facts, plans and screenshots. An editor reviews every change.

Claim it free Promote DeepEval

Sources

deepeval.com· checked 28 Sept 2026
deepeval.com/integrations· checked 28 Sept 2026
deepeval.com/docs/data-privacy· checked 28 Sept 2026
deepeval.com/enterprise· checked 28 Sept 2026
deepeval.com· checked 23 Sept 2026