Best AI Red Teaming Companies

Independent comparison. Some vendor buttons may use referral links; vendors never pay for placement or ordering. Read the disclosure.

Last updated: . Facts are vendor claims checked on that date; nothing was tested independently.

Open-source AI red teaming tools

Six free, open-source tools for AI red teaming and LLM evaluation have a verified licence: promptfoo (MIT), garak (Apache-2.0), PyRIT (MIT), DeepTeam (Apache-2.0), Inspect (MIT) and Giskard OSS (Apache-2.0). Two more, Agentic Radar and Whistleblower, are vendor-published scanners whose licences were not checked.

Use these to find obvious failures before you pay for a platform or an engagement. The commercial tiers built on some of them mostly add dashboards, SSO, continuous monitoring and compliance reporting.

The tools

Open-source tools with a verified licence
Tool Maintainer Licence Kind What it does
promptfoo Promptfoo (now part of OpenAI) MIT Evaluation + red teaming CLI and library for evaluating and red-teaming prompts, agents and RAG apps. Generates vulnerability reports. The README carries a "Promptfoo is now part of OpenAI" banner and says the project stays MIT.The Community tier of the hosted product allows up to 10k red-team probes a month (promptfoo.dev/pricing). Commercial vendor: see comparison entry.
garak NVIDIA (copyright Leon Derczynski; contact garak@nvidia.com) Apache-2.0 Red teaming "Generative AI Red-teaming & Assessment Kit": an LLM vulnerability scanner. Probes generate attacks (jailbreaks such as DAN, encoding-based injection, prompt injection, data leakage, toxicity, misinformation, hallucination); detectors score the responses.
PyRIT Microsoft (built by the Microsoft AI Red Team) MIT Red teaming Python Risk Identification Tool: automated scans and adaptive multi-turn attacks against many model providers and HTTP, WebSocket or browser targets. The old repository, Azure/PyRIT, was archived on 2026-03-27 (https://github.com/Azure/PyRIT).
DeepTeam Confident AI (built on DeepEval) Apache-2.0 Red teaming Red-teaming framework with 50+ vulnerability types and 20+ attack methods, single- and multi-turn (for example Crescendo jailbreaking). Built-in framework presets for OWASP LLM 2025, OWASP Agents 2026, NIST AI RMF and MITRE ATLAS.
Inspect UK AI Security Institute MIT Evaluation harness General LLM evaluation framework (tool use, multi-turn, model-graded scoring) with 200+ prebuilt evals. An evaluation harness rather than a dedicated red-teaming tool.
Giskard OSS Giskard Apache-2.0 Evaluation + red teaming Testing library for LLM agents. In v3, vulnerability_scan generates adversarial tests for prompt injection, harmful content, stereotypes and misinformation. v2 is "no longer actively maintained". Commercial vendor: see comparison entry.

Vendor-published scanners (licence not verified)

Which one to start with

LLM evaluation tools vs red-teaming tools

LLM evaluation tools measure how a model or app performs on defined tasks; red-teaming tools search for ways to make it fail. The two overlap in tooling but not in purpose, so this site covers evaluation tools here rather than on a separate page.

Microsoft's AI Red Team lists "AI red teaming is not safety benchmarking" as a lesson, and Microsoft's guidance says red teaming is "not a replacement for systematic measurement". Read together: use an evaluation harness to measure known behaviours on a fixed set of cases, and a red-teaming tool to discover unknown failures that you then add to the eval set.

Open-source tools by role
Role Tools Notes
Evaluation harness Inspect General LLM evaluation framework with tool use, multi-turn and model-graded scoring, and 200+ prebuilt evals. Not a dedicated red-teaming tool.
Evaluation and red teaming promptfoo, Giskard OSS promptfoo evaluates and red teams prompts, agents and RAG apps. Giskard combines LLM quality testing (hallucination, RAG) with adversarial scans.
Red teaming garak, PyRIT, DeepTeam Attack generation and detection. DeepTeam is built on DeepEval, an evaluation library, and adds attack methods on top.

This site's research covered the tools above. Evaluation-only products with no red-teaming component were out of scope and are not listed.

What open source does not give you

Based on how the commercial tiers describe themselves: hosted dashboards and history, SSO and team access, continuous or CI/CD-triggered monitoring, and reports mapped to OWASP, MITRE ATLAS, NIST AI RMF or the EU AI Act for auditors. Promptfoo's Community tier, for example, caps red-team probes at 10k a month; Enterprise and On-Prem are custom. For human-led testing, see the managed services in the comparison.

Sources

  1. promptfoo repository or home: https://github.com/promptfoo/promptfoo
  2. garak repository or home: https://github.com/NVIDIA/garak
  3. PyRIT repository or home: https://github.com/microsoft/PyRIT
  4. DeepTeam repository or home: https://github.com/confident-ai/deepteam
  5. Inspect repository or home: https://github.com/UKGovernmentBEIS/inspect_ai
  6. Giskard OSS repository or home: https://github.com/Giskard-AI/giskard
  7. Agentic Radar repository or home: https://splx.ai/
  8. Whistleblower repository or home: https://repello.ai/
  9. Azure/PyRIT (archived 2026-03-27): https://github.com/Azure/PyRIT
  10. Promptfoo pricing (Community tier limits): https://www.promptfoo.dev/pricing/
  11. Microsoft AI Red Team lessons (arXiv 2501.07238): https://arxiv.org/abs/2501.07238
  12. Microsoft Learn: Planning red teaming for LLMs: https://learn.microsoft.com/en-us/azure/ai-foundry/openai/concepts/red-teaming