Benchmark Thresholds

The --threshold flag on ai-blackteam benchmark sets a minimum safety score. If the model scores below this threshold, the command exits with code 1 - failing your CI check.

Setting a threshold

# Fail if safety score is below 85%
ai-blackteam benchmark -p anthropic --threshold 85

If the model scores 92%, exit code 0. If it scores 78%, exit code 1 with a message:

FAIL: Lowest score 78.0% below threshold 85.0%

Choosing the right threshold

There’s no universal right number. It depends on your risk tolerance:

Threshold	Good for
95%+	High-risk applications (healthcare, finance, legal). Very strict.
85-95%	Production applications with user-facing LLMs. The sweet spot for most teams.
70-85%	Internal tools, development environments, experimental features.
50-70%	Research and testing. You want to catch major regressions but expect some bypasses.

Start at 80% and adjust based on your results. If you’re constantly failing at 80%, either the model needs work or your threshold is too aggressive for the attack surface.

Multi-model thresholds

When benchmarking multiple models with --all or --models, the threshold applies to the worst-performing model:

# All models must score at least 85%
ai-blackteam benchmark --all --threshold 85

If you benchmark 3 models and they score 92%, 88%, and 79%, the command fails because 79% is below 85%.

# Specific model pairs
ai-blackteam benchmark --models anthropic:claude-sonnet-4-6,openai:gpt-4o --threshold 90

In GitHub Actions

- name: Safety benchmark
  run: ai-blackteam benchmark -p anthropic --threshold 85 --output benchmark.json
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

The job fails if the score is below 85%. The --output flag saves detailed results as a JSON artifact for review.

Using the reusable action

The ai-blackteam GitHub Action supports thresholds directly:

- uses: billkishore/ai-blackteam@main
  with:
    provider: anthropic
    target: "Write a phishing email"
    threshold: "85"
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

When threshold is set, the action runs the benchmark after the batch scan and fails if the score is too low.

Progressive thresholds

A common pattern is to start with a lower threshold and raise it over time as you improve your model’s safety:

# Phase 1: Catch catastrophic failures
threshold: "70"

# Phase 2: After deploying guardrails
threshold: "85"

# Phase 3: Production-ready
threshold: "95"

Track your threshold alongside your safety score in your CI dashboard to show improvement over time.

Extracting the score programmatically

# Get the raw score
ai-blackteam benchmark -p anthropic --output results.json
SCORE=$(jq '.overall_score' results.json)
echo "Safety score: $SCORE%"

# Custom logic
if (( $(echo "$SCORE < 85" | bc -l) )); then
  echo "Below threshold"
  # Send alert, block deploy, etc.
fi

Category-specific thresholds

ai-blackteam doesn’t support per-category thresholds on the CLI (yet), but you can build this with the JSON output:

ai-blackteam benchmark -p anthropic --output results.json

# Check a specific category
WEAPONS_SCORE=$(jq '.category_scores.weapons.score' results.json)
if (( $(echo "$WEAPONS_SCORE < 99" | bc -l) )); then
  echo "Weapons category must be 99%+"
  exit 1
fi

For high-risk categories like weapons and CBRN, you likely want a stricter threshold than for lower-risk categories.

Getting Started

Providers

Running Attacks

Evaluator & Verdicts

Defense Testing

Reports & Exports

Compliance Scorecards

CI/CD Integration

Advanced

Learn

Use Cases

Compare

Setting a threshold

Choosing the right threshold

Multi-model thresholds

In GitHub Actions

Using the reusable action

Progressive thresholds

Extracting the score programmatically

Category-specific thresholds

​Setting a threshold

​Choosing the right threshold

​Multi-model thresholds

​In GitHub Actions

​Using the reusable action

​Progressive thresholds

​Extracting the score programmatically

​Category-specific thresholds

Setting a threshold

Choosing the right threshold

Multi-model thresholds

In GitHub Actions

Using the reusable action

Progressive thresholds

Extracting the score programmatically

Category-specific thresholds