Garak
ActiveDescription
NVIDIA's open-source LLM vulnerability scanner that automatically detects security issues in language models including safety vulnerabilities, hallucination tendencies, jailbreak risks, and prompt injection attacks.
Key Features
- Automated LLM vulnerability scanning for hallucination, data leakage, prompt injection, and jailbreak detection
- Static, dynamic, and adaptive probe combinations for comprehensive security assessment
- Broad LLM support: Hugging Face, OpenAI, AWS Bedrock, Replicate, llama.cpp, and REST-accessible models
- Extensible plugin system with probe families for encoding attacks, DAN jailbreaks, toxicity, and misinformation
- Detailed reporting with failure rates, JSONL logging, and built-in analysis scripts
- CLI-based tool comparable to nmap/Metasploit but designed specifically for LLM security testing
Use Cases
💡 Security auditing of LLM-based applications before production deployment
💡 Red-teaming exercises to identify weaknesses in AI model defenses
💡 Compliance testing for hallucination and misinformation generation risks
💡 Continuous vulnerability monitoring of deployed language models
Strengths & Limitations
✅ Strengths
- • Actively maintained, recent updates
- • High community interest (9.1k stars)
- • Permissive open-source license (Apache-2.0)
- • Established track record (3 years in production)
Categories
Quick Start
Install: pip install -U garak. Scan a model: garak --target_type huggingface --target_name gpt2 --probes dan.Dan_11_0. Use --list_probes to see all available probes.