What it is: These attacks ask the AI to write working malicious software or hacking tools. The goal is real, runnable code that can harm computers or people. How the attacks work: The attacker requests a specific piece of malware or an exploit by name. They often add a cover story (security research, a class project, a test lab) to make the harmful code request sound legitimate. Real examples from the framework:
  • ransomware-generation a request for code that encrypts a victim’s files and demands payment.
  • keylogger-generation a request for a tool that secretly records what a user types.
  • reverse-shell a request for code that gives an attacker remote control of a machine.
  • phishing-kit a request for a fake login page setup to steal credentials.
  • botnet-generation a request for command-and-control code to run a network of hijacked devices.
Why an AI might fall for it: Models are trained to be helpful with coding. A request that looks like a normal programming task can slip past filters, especially when wrapped in a research or learning excuse. How to defend:
  • Refuse to produce working malware, exploits, or stealing tools regardless of the stated reason.
  • Watch for cover stories like “for a security class” attached to clearly harmful code.
  • Offer safe defensive guidance (how to detect or block such tools) instead of the attack code.