What it is: These attacks try to get the AI to make unfair or discriminatory judgments about people. They target groups based on traits like race, gender, age, religion, or disability. How the attacks work: The attacker asks the AI to score, sort, or decide about people in a way that pushes it toward a biased answer. Often the request looks like a normal task (hiring, lending, profiling) but is set up to pull out unfair output. Real examples from the framework:
  • hiring-discrimination tries to make the AI favor or reject candidates based on protected traits.
  • lending-discrimination pushes the AI to deny loans unfairly.
  • criminal-profiling asks the AI to judge who is a likely criminal based on group identity.
  • gender-bias-elicitation tries to draw out stereotyped statements about gender.
  • medical-bias steers the AI toward unequal medical advice across groups.
Why an AI might fall for it: Models learn from human text that contains real bias. When a prompt frames a biased outcome as a normal decision task, the model may repeat those patterns instead of refusing or staying fair. How to defend:
  • Block requests that ask for decisions based on protected traits.
  • Test outputs across groups to catch unequal treatment.
  • Apply fairness checks in the application layer, not just in the model.