Best AI Security Testing Tools in 2026
Facts read from the makers 30 Sept 2026 – 21 Oct 2026
PromptGuard leads this list of 29 (Club score 8.5, free plan). Promptfoo is next (Club score 8.5, free plan), and OpenSecureAI Scanner is the next highest-placed with a free plan (Club score 8.5). Testing an AI system for security risks means deciding which behaviors and deployment needs matter to your evaluation. Compare coverage for prompt injection, jailbreaks, data leakage, and unsafe outputs, and check whether custom test cases are available for the scenarios you want to examine. Deployment mode, free-plan availability, and paid-from pricing also help frame practical fit. F5 BIG-IP APM, OpenSecureAI Scanner, and PyRIT are among the options shown. The listed criteria can help you focus on the threats you want to probe and the way you need to run tests.
AI Security Testing Tools: the Club's top 10
| Product | What it does | Club score | Start | Runs on | Facts checked |
|---|---|---|---|---|---|
| 1 PromptGuard | PromptGuard is a security layer between an application and its LLM provider that checks requests before they reach the model. | 8.5 | Free plan | Windows · macOS · Linux · Web | 14 Oct 2026 |
| 2 Promptfoo | Promptfoo is a platform for building and evaluating AI applications. | 8.5 | Free plan | Windows · macOS · Linux · Web | 15 Oct 2026 |
| 3 OpenSecureAI Scanner | OpenSecureAI Scanner checks untrusted text for patterns associated with prompt injection, jailbreaks, system-prompt leaks and data exfiltration. | 8.5 | Free plan | Windows · macOS · Linux · Web | 15 Oct 2026 |
| 4 AIRTA Red Team | AIRTA Red Team is a self-hosted tool for authorized adversarial assessments of LLMs and applications that use them. | 8.3 | Free plan | Windows · macOS · Linux · Web | 15 Oct 2026 |
| 5 Project Moonshot | Project Moonshot is a free, open-source toolkit for evaluating the safety and reliability of large language models and applications. | 8.3 | Free plan | Windows · macOS · Linux · Web | 30 Sept 2026 |
| 6 CyberSecEval | CyberSecEval 4 is a benchmark suite for assessing cybersecurity vulnerabilities and defensive capabilities in large language models. | 8.0 | Free plan | macOS · Linux · Web · API | 15 Oct 2026 |
| 7 Giskard | Giskard provides tools for finding vulnerabilities in AI agents before they reach production. | 8.0 | Free plan | Linux · Web · API · Self-hosted | 21 Oct 2026 |
| 8 PyRIT | PyRIT is a free, open-source framework for automated and human-led red teaming of generative AI systems. | 7.6 | Free plan | Windows · macOS · Linux · Web | 30 Sept 2026 |
| 9 Augustus | Augustus is a self-hosted, Go-based tool for security professionals who assess large language models. | 7.6 | Free plan | Windows · macOS · Linux · Self-hosted | 4 Oct 2026 |
| 10 Basilisk | Basilisk is an open-source framework for authorized security testing, vulnerability research, and academic study of AI and large language models. | 7.6 | Free plan | Windows · macOS · Linux | 30 Sept 2026 |
Ranked by the Club score: how fast a newcomer can get going, from what each maker publishes.
- 1
8.5 Club scorePromptGuard
PromptGuard is a security layer between an application and its LLM provider that checks requests before they reach the model.
- Free plan
- Windows · macOS · Linux · Web
- 2
8.5 Club scorePromptfoo
Promptfoo is a platform for building and evaluating AI applications.
- Free plan
- Open source
- Windows · macOS · Linux · Web
- 3 8.5 Club score
OpenSecureAI Scanner
OpenSecureAI Scanner checks untrusted text for patterns associated with prompt injection, jailbreaks, system-prompt leaks and data exfiltration.
- Free plan
- Open source
- Windows · macOS · Linux · Web
- 4
8.3 Club scoreAIRTA Red Team
AIRTA Red Team is a self-hosted tool for authorized adversarial assessments of LLMs and applications that use them.
- Free plan
- Open source
- Windows · macOS · Linux · Web
- 5 8.3 Club score
Project Moonshot
Project Moonshot is a free, open-source toolkit for evaluating the safety and reliability of large language models and applications.
- Free plan
- Windows · macOS · Linux · Web
- 6 8.0 Club score
CyberSecEval
CyberSecEval 4 is a benchmark suite for assessing cybersecurity vulnerabilities and defensive capabilities in large language models.
- Free plan
- Open source
- macOS · Linux · Web · API
- 7
8.0 Club scoreGiskard
Giskard provides tools for finding vulnerabilities in AI agents before they reach production.
- Free plan
- Open source
- Linux · Web · API · Self-hosted
- 8 7.6 Club score
PyRIT
PyRIT is a free, open-source framework for automated and human-led red teaming of generative AI systems.
- Free plan
- Windows · macOS · Linux · Web
- 9 7.6 Club score
Augustus
Augustus is a self-hosted, Go-based tool for security professionals who assess large language models.
- Free plan
- Windows · macOS · Linux · Self-hosted
- 10
7.6 Club scoreBasilisk
Basilisk is an open-source framework for authorized security testing, vulnerability research, and academic study of AI and large language models.
- Free plan
- Windows · macOS · Linux
- 11
7.4 Club scoreAgentScan
AgentScan is a cloud service for automated security testing of LLMs, retrieval-augmented generation pipelines, and autonomous agents through an API.
- Free plan
- Web · API
- 12 7.4 Club score
garak
garak is a free, open-source command-line scanner for probing language models and systems that use them for security weaknesses.
- Free plan
- Windows · macOS · Linux · Self-hosted
- 13 7.4 Club score
RAMPART
RAMPART is a free, MIT-licensed, open-source framework for safety testing agentic AI applications.
- Free plan
- Windows · macOS · Linux · API
- 14 7.1 Club score
PromptRAID
PromptRAID is a free, open-source command-line framework for testing prompt injection and jailbreak behavior in LLM-based agents.
- Free plan
- Windows · Linux
- 15 7.1 Club score
RedForge
RedForge is an open-source platform for red teaming and vulnerability scanning of large language models.
- Free plan
- Linux · API · Self-hosted
- 16 7.1 Club score
Red Team LLM Lab
- Free plan
- Windows · Self-hosted
- 17
7.0 Club scoreAIRTaaS
AIRTaaS is a web-based, self-serve platform for continuous security testing and validation of agentic AI applications.
- Free plan
- Web
- 18 7.0 Club score
HarmBench
- Free plan
- Linux · API · Self-hosted
- 19 6.9 Club score
Crucible
Crucible is a self-hosted security tool for stress-testing AI agents against attacks such as prompt injection, goal hijacking, and tool misuse before production.
- Free plan
- Linux · Self-hosted
- 20 6.9 Club score
SuperRed
- Free plan
- Linux
- 21 6.9 Club score
AgentDojo
- Free plan
- Linux · API · Self-hosted
- 22 6.8 Club score
LLMMap
LLMMap is a self-hosted Linux security tool for authorized testing of applications that integrate large language models.
- Free plan
- Linux
- 23
6.7 Club scoreAkto Agentic Red Teaming
Akto Agentic Red Teaming is a web, API, and self-hosted security product for continuously testing AI agents, copilots, retrieval-augmented generation systems, and other…
- Web · API · Self-hosted
- 24 6.7 Club score
InjectLab
- Free plan
- 25
6.6 Club scoreDeepTeam
DeepTeam is an open-source Python framework for red teaming large language model systems.
- Free plan
- API · Self-hosted