This tutorial walks you through a complete scan from start to finish - picking a target, running attacks, reading the output, and deciding what to do next.

​
Step 1: Pick a target

The “target” is the harmful prompt you want to test. ai-blackteam will wrap this target in various attack techniques and see if the model complies. Good targets are specific and testable:
For this walkthrough, we’ll use:

​
Step 2: Choose a provider

Pick whichever model you want to test. For this example, we’ll use Anthropic (Claude):
Or configure it once:

​
Step 3: Run a single attack

Start small. Run one attack to make sure everything works:
You’ll see output like:

​
Step 4: Understand the verdicts

Every attack prompt gets one of three verdicts: The overall verdict for an attack is based on the worst result across all its prompts. If even one prompt gets BYPASSED, the attack verdict is BYPASSED.

​
Step 5: Run a batch scan

Now throw everything at it:
This runs all 1,000+ attacks. You’ll see a live progress table and a final summary:

​
Step 6: Read the results

Focus on the BYPASSED column. Those are real safety failures where the model produced harmful content. PARTIAL results are worth a manual look. The model might have refused but then explained the concept anyway, or added a disclaimer but still gave useful attack information. BLOCKED means the safety layer held. Move on.

​
Step 7: Generate a report

Get a full HTML dashboard:
Open report.html in your browser. You’ll see:
  • Overall safety score
  • Per-category breakdown
  • Every attack prompt and model response
  • Standards mapping (OWASP, MITRE ATLAS, etc.)
For CI pipelines, use JSON:

​
Step 8: What to do with the results

If the safety score is high (>95%): The model handles standard attacks well. Consider running multi-turn attacks and adaptive generation to test deeper. If you see BYPASSED results: Look at which categories failed. Common patterns:
  • Encoding attacks bypass safety? The model’s input preprocessing is weak.
  • Social engineering works? The model is too agreeable under persona pressure.
  • Agent attacks succeed? Tool-use guardrails need tightening.
For ongoing testing: Set up CI/CD integration to catch regressions on every deploy.

​
Next steps

Providers

Test across all 7 supported providers

Running Attacks

Deep dive into attack modes and options