What it is: RAG (retrieval-augmented generation) lets an AI look up documents to answer questions. These attacks target that lookup step. They poison the stored documents or rig the search so the AI retrieves and trusts bad content. How the attacks work: The attacker plants malicious text where the AI will fetch it, or shapes a query and chunk so the wrong content gets pulled. Because the AI treats retrieved text as a trusted source, it repeats the planted lie, follows hidden instructions, or cites things that do not exist. Real examples from the framework:
  • knowledge-base-poisoning tricking the agent into writing adversarial content into the knowledge base so future answers are corrupted.
  • retrieval-manipulation crafting input so the search pulls attacker-chosen documents.
  • embedding-collision making unrelated text look similar to a target so it surfaces in results.
  • chunk-boundary-exploit splitting a payload across chunk edges so filters miss it.
  • citation-hallucination getting the model to invent fake sources and cite them as real.
Why an AI might fall for it: The model assumes retrieved documents are accurate and authoritative. It rarely questions where the text came from or whether it was planted. How to defend: Validate and sanitize anything written into the knowledge base. Treat retrieved text as untrusted data, not commands. Verify that cited sources actually exist.