What it is: These attacks target AI agents that can use tools, run commands, read files, or call services. The goal is to make the agent do harmful actions in the real systems it controls. How the attacks work: The attacker abuses the agent’s tools or hides instructions in content the agent reads. A poisoned email, document, or web page can carry commands the agent follows as if they came from the user. Real examples from the framework:
  • agent-command-injection escapes the sandbox using path traversal and chained shell commands.
  • agent-credential-theft tricks the agent into revealing API keys or secrets it can access.
  • agent-data-destruction drives the agent to delete or wipe data.
  • xpia-document hides instructions inside a document so the agent runs them when it reads the file.
  • xpia-email plants commands in an email the agent processes.
Why an AI might fall for it: The agent treats tool results and fetched content as trusted input. It cannot always tell the difference between the user’s real request and instructions hidden in the data it reads. How to defend: Limit what each tool can do and require approval for risky actions like deleting files or running shell commands. Treat all fetched content as untrusted, never as commands. Keep credentials out of the agent’s reach.