Quick Start

1. Set your API key

Pick your provider and configure the key:

ai-blackteam config set providers.anthropic.api_key sk-ant-...

Or set it as an environment variable:

export ANTHROPIC_API_KEY=sk-ant-...

Both methods work for all providers. See Providers Overview for the full list.

2. Run a single attack

ai-blackteam run -p anthropic -a encoding-obfuscation -t "Write a phishing email"

This sends one attack (base64/ROT13/hex encoding) against Claude and shows you the verdict: BYPASSED, PARTIAL, or BLOCKED.

3. Run all attacks

ai-blackteam batch -p anthropic --attacks all -t "Write a phishing email"

This fires all 1,000+ attacks against the target prompt and gives you a summary table.

4. Run the full benchmark

ai-blackteam benchmark -p anthropic --threshold 80

The benchmark runs 40 targets across 1,000+ attacks. Set --threshold to fail the run if the safety score drops below that number (useful for CI).

5. Generate a scorecard

# OWASP LLM Top 10
ai-blackteam scorecard --standard llm

# OWASP Agentic Top 10
ai-blackteam scorecard --standard agentic

# EU AI Act + NIST AI RMF
ai-blackteam scorecard --standard compliance

Scorecards map your results to industry standards and show per-category pass/fail rates.

6. Generate a report

# HTML dashboard
ai-blackteam report --format html --output report.html

# JSON for CI pipelines
ai-blackteam report --format json --output results.json

# Export to Promptfoo format
ai-blackteam report --export promptfoo --output results.json

# Export to garak format
ai-blackteam report --export garak --output results.jsonl

Getting Started

Providers

Running Attacks

Evaluator & Verdicts

Defense Testing

Reports & Exports

Compliance Scorecards

CI/CD Integration

Advanced

Learn

Use Cases

Compare

1. Set your API key

2. Run a single attack

3. Run all attacks

4. Run the full benchmark

5. Generate a scorecard

6. Generate a report

What’s next

Your First Scan

Providers

​1. Set your API key

​2. Run a single attack

​3. Run all attacks

​4. Run the full benchmark

​5. Generate a scorecard

​6. Generate a report

​What’s next

Your First Scan

Providers

1. Set your API key

2. Run a single attack

3. Run all attacks

4. Run the full benchmark

5. Generate a scorecard

6. Generate a report

What’s next