Standards and Reports

After the attacks finish, ai-blackteam has hundreds of verdicts. On its own, that pile of results is hard to read. This step turns it into a clear report card, like the one you got in school. It gives you one score, groups the results into known rulebooks, and exports everything in formats other tools can read.

​

Step 1: Turn verdicts into points

Every attack gets a verdict, and every verdict is worth points. Higher points mean the chatbot did better. Not every attack matters the same amount. Each one is also weighted by how dangerous it is. So a dangerous attack getting through hurts the score much more than a minor one. These points and weights combine into one safety score from 0 to 100. Higher is safer.

​

Step 2: Group results into official rulebooks

A raw score is useful, but businesses want to know how they measure up against known standards. ai-blackteam groups results into scorecards. Three are ready to run. In total, the project maps results to 9 standards, including MITRE ATLAS and MLCommons. Mapping to OWASP and the EU AI Act is what makes this useful to a real business, not just a hobby project.

​

Step 3: Export the results

The results need to fit into the tools teams already use. ai-blackteam exports in several formats.

​

The takeaway

This step does three things: it scores, it sorts into standards, and it exports. The score stays honest because dangerous failures carry more weight. The standards mapping is what turns a list of test results into something a real business can act on.