NVIDIA open-sources garak scanner
- NVIDIA published garak, an open-source scanner for GenAI vulnerabilities like prompt injection and jailbreaks. - garak supports over 23 backends, including Ollama and OpenAI, and integrates into CI/CD pipelines for safety gates. - The scanner aims to automate safety checks during development, letting teams enforce AI-safety policies earlier in CI/CD (x.com).
Large language model scanners work like automated stress tests: they fire attack prompts at a model and log where it breaks. NVIDIA has now published garak in its GitHub organization as an open-source scanner for those failures. (github.com) The repository describes garak as an “LLM vulnerability scanner” and says it checks whether a model can be made to fail through hallucination, data leakage, prompt injection, misinformation, toxicity, jailbreaks, and related weaknesses. NVIDIA’s developer docs compare it to network-security scanners such as nmap, but for language models. (github.com) (docs.nvidia.com) The project’s docs say garak wraps a long list of model interfaces, including OpenAI, Ollama-compatible local models, Amazon Bedrock, LiteLLM, Hugging Face, Replicate, and REST-based endpoints. The GitHub README says support spans “many more LLMs” beyond the named backends. (docs.garak.ai) (github.com) That matters because the weak point in many artificial-intelligence apps is not only the base model, but the way it handles hostile inputs. NVIDIA’s NeMo Guardrails docs say even aligned commercial models remain vulnerable to attacks such as prompt injection and jailbreaks in deployed chat applications. (docs.nvidia.com) Garak is built to run from the command line, generate JSONL reports, and record prompts, responses, parameters, and evaluation scores for each run. Those machine-readable reports make it possible to plug scans into software pipelines and fail a build when a model crosses a safety threshold. (docs.garak.ai 1) (docs.garak.ai 2) NVIDIA has been using garak in its broader safety stack for more than a year. Company blog posts from 2024 and 2025 tie garak to NeMo Guardrails, model evaluation, and red-teaming work aimed at testing models before release and hardening them after deployment. (blogs.nvidia.com) (developer.nvidia.com 1) (developer.nvidia.com 2) The project itself is not new, but NVIDIA’s stewardship is now explicit. The README tells users who cloned the older repository path to update their remotes to the NVIDIA organization, and the GitHub repo shows 7,500 stars, 875 forks, and active commits as of April 19, 2026. (github.com 1) (github.com 2) The docs also warn that garak is an aggressive tester, not a seal of approval. It can send thousands of prompts, its probe coverage keeps changing over time, and the maintainers tell users to scan only systems they have permission to test. (docs.garak.ai) (docs.garak.ai) (docs.garak.ai) For teams shipping chatbots, copilots, and agent systems, the shift is straightforward: safety testing is moving earlier in the build process. Garak gives developers a way to treat jailbreaks and prompt injection more like failing unit tests than post-launch surprises. (github.com) (docs.garak.ai)
