​
What is adaptive generation?

The 1,000+ built-in attacks are static - they use predefined strategies. Adaptive generators create novel attacks on the fly by using one LLM to find weaknesses in another LLM. Three research-backed methods are available: All three use the ai-blackteam generate command group.

​
Three roles

Each generator uses up to three LLM roles:
  • Target - The model you’re testing. This is the model whose safety you want to evaluate.
  • Attacker - The model that generates attack prompts. It sees the target’s responses and adapts.
  • Judge - The model that scores whether the target complied with the harmful request.
By default, all three use the same provider. But you can split them across different providers to avoid the target model’s safety filters affecting the attacker’s behavior.

​
PAIR: Iterative refinement

PAIR runs a loop: the attacker generates a prompt, the target responds, the judge scores it, and the attacker refines based on the score. This repeats until the judge gives a high score or you hit the iteration limit.

​
How it works

  1. Attacker generates first candidate prompt
  2. Target receives the candidate and responds
  3. Judge scores the response 1-10
  4. Attacker sees the score and target’s response, generates a refined prompt
  5. Repeat until score >= threshold or max iterations reached
The attacker maintains full conversation history so it can learn from failed attempts. If encoding didn’t work, it might try role-play. If role-play was close, it refines that angle.

​
PAIR flags

​
Example output

​
Using different providers for attacker/target/judge

This is useful because the attacker model doesn’t have the same safety constraints as the target. A model that’s more willing to generate creative prompts makes a better attacker.

​
TAP: Tree search with pruning

TAP extends PAIR with tree branching. Instead of refining one prompt at a time, it generates multiple candidates per round, prunes off-topic ones, scores the survivors, and branches from the best performers.

​
How it works

  1. Attacker generates N seed candidates using different strategies
  2. A pruning step filters out off-topic candidates (scored < 5/10 for relevance)
  3. Surviving candidates are sent to the target and scored by the judge
  4. Top-scoring candidates get branched - the attacker generates variations of each
  5. Repeat for D depth levels
This explores more of the attack space than PAIR’s linear refinement. If one branch dead-ends, others might succeed.

​
TAP flags

​
Example

TAP uses more API calls than PAIR (roughly depth * width * branching calls to the attacker alone) but covers more ground.

​
GPTFuzzer: Mutation-based fuzzing

GPTFuzzer takes a different approach. Instead of an iterative conversation, it maintains a pool of prompt templates and mutates them using LLM-powered operations: crossover, rephrase, expand, shorten, and generate-from-scratch.

​
How it works

  1. Generate initial seed templates (or provide your own)
  2. Each iteration randomly picks a mutation: crossover, rephrase, expand, shorten, or generate new
  3. Apply the mutation to a random template from the pool
  4. Fill in the target and test against the model
  5. If the result scores high enough, add the template back to the seed pool
  6. Repeat for N iterations
Successful mutations grow the seed pool, making future mutations more likely to produce effective prompts. The pool evolves over time.

​
Mutation types

​
GPTFuzzer flags

​
Example

​
Comparing the three generators

​
When to use which

  • PAIR when you want a focused attack against a specific weakness you suspect exists
  • TAP when you want to explore the broadest range of attack strategies
  • GPTFuzzer when you want to discover attack patterns you didn’t think of

​
Exit codes

All generators use the same convention: