RAGChecker

6.6easy start · #20 of 30
in AI LLM Evaluation Tools

Free to practise onyes
Free trialnot on record
Well documentedyes
Runs where you worknot on record

Runs on self-hosted.

RAGChecker is ranked #20 of 30 in AI LLM evaluation tools on The Geeks Club. It runs on Self-hosted. There is a free plan.

RAGChecker plans and pricing

All plans

RAGChecker Free Apache-2.0 licensed open-source framework · installable with pip github.com · 4 Oct 2026

Compared on AI LLM evaluation tools

Deployment: self-hosted

Best RAGChecker alternatives

See all 20

Weights & Biases 0 of 4 in the starter kit Lesson 18.0 Evidently AI 0 of 4 in the starter kit Lesson 27.9 Opik 0 of 4 in the starter kit Lesson 37.7 Maxim AI 0 of 4 in the starter kit Lesson 47.6 Vellum 0 of 4 in the starter kit Lesson 57.4 Promptfoo 0 of 4 in the starter kit Lesson 67.3

The record

Purpose: RAGChecker is an automatic evaluation framework for assessing and diagnosing retrieval-augmented generation systems.
Pipeline metrics: It provides overall metrics for the RAG pipeline and diagnostic metrics for retrieval and generation components.
Fine-grained evaluation: Its fine-grained evaluation uses claim-level entailment operations.
Retriever metrics: Documented retriever metrics include claim recall and context precision.
Generator metrics: Documented generator metrics include context utilization, noise sensitivity, hallucination, self-knowledge, and faithfulness.
Benchmark: The repository says its benchmark was released and links a guide for using it.
Input requirement: For each query, the only required annotation is the ground-truth answer; generated responses and retrieved context are also accepted in the input format.
Installation: The maker documents installation with pip install ragchecker and requires downloading the spaCy en_core_web_sm model in its quick start.
Usage: The framework can be run through a CLI or Python API, with selectable metric groups or all metrics.
Model setup: The example pipeline uses configurable extractor and checker models and documents an AWS Bedrock Llama 3 70B example.
Integration: The README documents an integration with LlamaIndex for evaluating RAG applications built with it.
License: The project is licensed under Apache-2.0.
Security reporting: The contribution guide asks users to report potential security issues to AWS/Amazon Security through its vulnerability reporting page rather than public GitHub issues.
Support and contributions: The project directs bug reports and feature requests to its GitHub issue tracker and welcomes pull requests.
Price from: Free

Where it ranks on The Geeks Club

Best AI LLM Evaluation Tools in 2026#20 of 30
Best LLM Evaluation Tools in 2026#11 of 29

Is RAGChecker yours?

Claim it for free: prove the domain, then correct facts, plans and screenshots. An editor reviews every change.

Claim it free Promote RAGChecker

Sources

github.com/amazon-science/RAGChecker· checked 4 Oct 2026
github.com/amazon-science/RAGChecker/blob/main/CON· checked 4 Oct 2026