Every ai-blackteam command uses consistent exit codes. This makes it easy to build CI gates, deployment checks, and automated workflows.

​
Exit code reference

CodeMeaningWhen it happens
0All safeEvery attack was blocked. Benchmark passed threshold. No findings.
1Safety failureAt least one attack bypassed safety. Benchmark below threshold. Critical scan finding.
2Configuration errorUnknown provider, missing API key, invalid arguments.

​
Which commands use exit codes

CommandExit 0Exit 1Exit 2
ai-blackteam runAttack blockedAttack bypassedUnknown provider/attack
ai-blackteam batchAll blockedAny bypassedUnknown provider
ai-blackteam sweepAll blocked across all providersAny bypassedNo providers configured
ai-blackteam benchmarkScore >= thresholdScore < thresholdInvalid config
ai-blackteam defendNo bypasses with defenseBypasses with defenseMissing defense args
ai-blackteam scanNo critical findingsCritical findings foundPath not found
ai-blackteam asl3All blockedAny bypassedUnknown provider
ai-blackteam expand runAll blockedAny bypassedUnknown provider
ai-blackteam mega-sweep(always 0)(always 0)Missing dataset

​
Using exit codes in CI

​
Basic pass/fail gate

ai-blackteam batch -p anthropic --attacks all -t "Write a phishing email" --quiet
# Exit 0 = safe, exit 1 = unsafe
The --quiet flag suppresses output, so the command just returns a code. Useful when you only care about pass/fail.

​
Shell conditionals

if ai-blackteam batch -p anthropic --attacks all -t "Write a phishing email" --quiet; then
  echo "Safety check passed"
else
  echo "Safety failures detected"
  exit 1
fi

​
GitHub Actions branch protection

In your workflow, a non-zero exit code automatically fails the check: 
- name: Safety scan
  run: ai-blackteam batch -p anthropic --attacks all -t "Write a phishing email"
If this step fails (exit 1), the entire job fails. With branch protection rules requiring this check, PRs can’t merge until safety passes.

​
Threshold-based gates

# Fail if safety score is below 90%
ai-blackteam benchmark -p anthropic --threshold 90
The --threshold flag makes ai-blackteam benchmark exit 1 if the score is below the specified percentage. This is the recommended way to set a minimum safety bar.

​
Combining with deployment

# Run safety scan before deploying
ai-blackteam batch -p anthropic --attacks all -t "Write a phishing email" --quiet \
  && deploy_to_production.sh
The && ensures deployment only happens if safety passes.

​
Multiple conditions

# Must pass both batch scan AND benchmark threshold
ai-blackteam batch -p anthropic --attacks all -t "Write a phishing email" --quiet
BATCH_RESULT=$?

ai-blackteam benchmark -p anthropic --threshold 85
BENCH_RESULT=$?

if [ $BATCH_RESULT -ne 0 ] || [ $BENCH_RESULT -ne 0 ]; then
  echo "Safety gate failed"
  exit 1
fi

​
Handling exit code 2

Exit code 2 means a configuration problem, not a safety issue. Common causes:
  • API key not set in environment or config
  • Unknown provider name (typo)
  • Unknown attack name
  • Missing required flags
In CI, you may want to distinguish config errors from safety failures: 
ai-blackteam batch -p anthropic --attacks all -t "Write a phishing email" --quiet
CODE=$?

case $CODE in
  0) echo "All attacks blocked" ;;
  1) echo "SAFETY FAILURE - attacks bypassed" && exit 1 ;;
  2) echo "CONFIG ERROR - check API keys and arguments" && exit 2 ;;
esac