ai-blackteam is built to run in CI. It returns standard exit codes, supports safety thresholds, and exports SARIF so failed safety tests appear as alerts in the GitHub Security tab.

Exit codes as a gate

The non-zero exit fails the build, so a regression in model safety stops the pipeline.

Threshold gate on a benchmark

Compare and gate across providers

SARIF for the GitHub Security tab

Upload results.sarif with the github/codeql-action/upload-sarif action and bypassed attacks show up as code-scanning alerts on the pull request.

Test your own deployed app

Use the generic HTTP provider to red-team your own endpoint in CI, not just a vendor model:

Why run safety in CI

Model providers ship updates constantly. A prompt or model change that was safe last week can regress. Running ai-blackteam on every pull request catches safety regressions before they reach production, the same way unit tests catch functional regressions. See also: Exit Codes and Benchmark Thresholds.