How the pieces fit together
The same picture with kitchen-restaurant labels for the non-technical reader:| Component | Role | Restaurant analogy |
|---|---|---|
| CLI / Python API | Entry point: parses your command, dispatches to the engine | The waiter taking your order |
| Engine + Evaluator | Orchestrates the attack, scores the response | The kitchen cooking and tasting |
| Attacks (plugins) | 1,020 curated techniques + 19 dataset loaders + 7 adaptive generators | The ingredients |
| Providers (plugins) | 17 integrations (16 vendors + generic HTTP) plus a mock | The stoves |
| SQLite | Per-run + per-turn audit trail | The receipt book |
| Reports / Scorecards | Roll-ups into OWASP / MITRE / MLCommons formats | The customer reviews |
Project Stats
| Metric | Count |
|---|---|
| Curated attack techniques | 1,020 |
| Public benchmark loaders | 19 |
| Adaptive generators | 7 |
| Tests | 3,059 |
| Standards mapped | 9 |
| Categories | 61 |
| Providers | 17 |
| Attack surface | 163M configurations |
| Version | 1.7.1 |
Attack surface
The framework multiplies a small set of building blocks (1,020 techniques, plus prompts loaded from 19 public benchmarks) across harm categories, difficulty levels, mutations, and languages to reach a search space of about 163 million configurations. You sample from it, you never run it all. The full breakdown and exact math live in The 163 Million Attack Surface.Design philosophy
Five plugin contracts power everything. Each is an abstract base class, so adding a new one is a single Python file with no changes elsewhere:| Contract | What it guarantees |
|---|---|
BaseAttack | Every attack generates prompts for any target |
BaseProvider | Every provider sends prompts and returns responses |
BaseGenerator | Every adaptive generator rewrites and escalates prompts from a target’s responses |
DatasetLoader | Every loader pulls and normalizes a public benchmark dataset |
BaseVerifier | Every verifier turns a response into a verdict |