Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

garak

garak.ai

  • Free plan
  • Windows
  • macOS
  • Linux
  • Self-hosted
7.4 Club score Visit garak.ai

What garak is

garak is a free, open-source command-line scanner for probing language models and systems that use them for security weaknesses. Its probes cover issues including hallucination, data leakage, prompt injection, misinformation, toxicity generation, and jailbreaks. Probes attempt to elicit failures, while detectors inspect responses for problems such as unsafe content or failure to refuse.

A standard scan can run without supervision and produce a report, with scan configuration available for customization. Supported connections include OpenAI, Hugging Face, Cohere, Replicate, AWS Bedrock, LiteLLM, REST-accessible models, and custom Python integrations. Hugging Face models can run locally or be accessed through an inference API or private endpoint. Results include screen output, a detailed report log, a hit log, and a debug log.

The reference documentation describes support for mainstream any-to-any large models, but assessments cover text output only. garak is developed in Linux and OSX and can be installed with pip or from source; deployment is self-hosted. Listed capabilities include custom metrics, LLM-as-a-judge, and safety evaluations. The project is backed by NVIDIA and the community, and points users to guides, Discord, GitHub issues, and an NVIDIA email address for support.

Who it suits

garak suits teams and developers who want to scan language models for security weaknesses and can work with a command-line tool. It is intended for self-hosted deployment and evaluates text output.

Good to know, and what to watch

  • Free and open-source.
  • Scans for prompt injection and data leakage.
  • Supports multiple model providers and custom integrations.
  • Can run scans without supervision.
  • Provides detailed, hit, and debug logs.
  • Assesses text output only.
  • Requires command-line installation or source setup.
  • Deployment is self-hosted.

Free

Free

Free plan

The verdict

garak offers configurable probes, model integrations, and several report outputs for evaluating language-model weaknesses. Its text-only assessment scope and self-hosted command-line setup are important constraints to weigh.

What the maker publishes

Automated scans
garak can run a standard scan and produce a report without supervision, while allowing scan configuration to be customized. source
Deployment
garak is a command-line tool developed in Linux and OSX and can be installed with pip or from source. source
Integrations
The project lists support for OpenAI, Hugging Face, Cohere, Replicate, AWS Bedrock, LiteLLM, REST-accessible models, and custom Python integrations. source
Local and hosted models
The documentation describes running Hugging Face models locally or accessing them through an inference API or private endpoint. source
Maintainer
The project website says garak is backed by NVIDIA and is updated by NVIDIA and the community. source
Output scope
The reference documentation says garak supports mainstream any-to-any large models but assesses text output only. source
Probe and detector design
Probes attempt to elicit failures, and detectors examine model outputs for failure modes such as unsafe content or failure to refuse. source
Purpose
garak is an LLM vulnerability scanner that probes language models and systems using them for security weaknesses. source
Reports
garak produces screen output, a detailed report log, a hit log, and a debug log. source
Security reporting
The repository directs security vulnerability reports to NVIDIA’s security submission form or [email protected] and says not to report them through GitHub. source
Support
The project points users to its user guide, reference documentation, Discord, GitHub issues, and an NVIDIA email address for support. source
Threat coverage
The project lists hallucination, data leakage, prompt injection, misinformation, toxicity generation, and jailbreaks among the weaknesses it probes. source

Checked against the maker's pages on 7 Oct 2026.