RAG Knowledge-Base Poisoning: When Retrieved Evidence Is Manipulated

A support chatbot answers a customer's question with total confidence. The answer cites a document from the company knowledge base. The only problem: the document was planted by someone who wanted the chatbot to say exactly that.

This is the uncomfortable truth about retrieval-augmented generation. Grounding answers in evidence sounds like a safety feature—and it is, compared to letting a model answer from memory alone. But grounding is not the same as trust. Retrieval selects documents by semantic similarity, not by truthfulness. Anything that gets indexed can become "evidence."

Why Grounded Answers Can Still Be Wrong

Think back to the RAG pipeline: documents get ingested, chunked, embedded, and stored in a vector database. When a user asks a question, the system embeds that query and retrieves the chunks whose embeddings are most similar. Those chunks go into the context window, and the model writes an answer grounded in them.

Here is the gap: the default similarity search does not check whether a document is true, authorized, or safe. Retrieval ranks by similarity. A document that is semantically close to the query wins, regardless of whether it came from your legal team or from someone who crafted it to look like it did.

That is what makes rag data poisoning so effective. An attacker does not need to break into your model or manipulate your prompt. They just need to get a document into the knowledge base that scores high for the queries they want to influence. Once retrieved, that text becomes influential context—unless your system enforces a stronger boundary through explicit ingestion, access, validation, or generation controls.

This is distinct from prompt injection, where an attacker sneaks instructions into the live input stream. Knowledge-base poisoning corrupts the source corpus itself. The poisoned content sits in the index, waiting to be retrieved. The attack happens upstream, before the user ever asks a question.

How Poisoned Documents Get Retrieved

Imagine a bank's customer-support chatbot. A user asks, "How do I verify my account?" The legitimate answer involves uploading an ID and waiting for review. But an attacker has inserted a document that looks like an official security notice. It uses the same terminology as real bank communications: "account verification," "identity confirmation," "security protocol." It is formatted like an internal update, complete with urgent headers.

When the user's query gets embedded, that poisoned document scores high similarity—perhaps higher than the legitimate one. It lands in the context window. The model reads it as grounded evidence and produces an answer that sends the user to a phishing page or reveals the wrong verification steps.

The attack is efficient because the poisoned document does the work of ranking itself. Attackers stuff documents with keyword clusters that match target queries and format them to look authoritative. Research on RAG poisoning has shown that injecting a small number of malicious texts into a knowledge database can steer model outputs toward attacker-chosen answers. The effect is not localized, either. One poisoned document can shape answers for every query that retrieves it.

Where Poisoning Enters the Pipeline

Before listing entry points, one boundary matters. Knowledge-base poisoning means untrusted or manipulated content gets admitted to the retrievable corpus. Three adjacent risks share the neighborhood but are not the same failure:

• Authorization failures let the wrong person read the right document.
• Retriever manipulation tricks the search step into selecting bad content without modifying the corpus.
• Prompt injection inside a document smuggles instructions into the context window through retrieved text.

This article covers the first case: content that should never have been trusted as evidence, entering the index where retrieval can find it. The other three are neighboring risks with their own controls.

With that boundary set, here are the common entry points.

Ingestion from external sources. If your pipeline scrapes websites, pulls from shared drives, or imports vendor documentation, every source is a potential vector. An attacker who can publish content to a site you scrape—or convince an employee to upload a file—has a foothold.

Direct insertion into the vector database. Anyone with write access to the index can insert documents that bypass normal ingestion checks. This is the most direct form of knowledge base poisoning rag, and it is why access control on the database itself matters.

Manipulated metadata. Attackers do not always need to plant a whole document. Changing tags, titles, or other metadata can alter which documents get retrieved for which queries. A legitimate document about refunds can be retagged to surface for security questions.

Synthetic content reintroduced without attribution. If your pipeline generates summaries or answers and then feeds them back into the corpus, you can create a feedback loop where model output becomes future "evidence"—including output that was shaped by poisoned content in the first place.

Shared documents without verification. In many organizations, files enter the knowledge base automatically when shared to a certain folder or uploaded to a portal. No human reviews them. No provenance is recorded.

Why This Is Hard to Spot

Poisoned documents are designed to blend in. They look like legitimate sources because they imitate them. They use the right vocabulary, the right formatting, the right tone. Nothing about them triggers a human reader's suspicion.

The model makes it worse. It produces confident, fluent answers regardless of whether the underlying evidence is sound. A poisoned answer does not arrive with error flags or hesitation. It arrives with the same polished prose as a correct one.

This is why the failure is so easy to misattribute. When a RAG system gives a wrong answer, the natural instinct is to blame the model. But the model may be doing exactly what it was designed to do: faithfully summarizing the evidence it was given. The fault lies in the evidence, not the generator.

Detection is an investigation, not a magic filter. Statistical anomaly detection and basic pattern matching rarely catch sophisticated attacks, because a document that mimics your legitimate corpus is not statistically anomalous. Instead, look for concrete signals: documents without recorded provenance, chunks that appear across unrelated queries, content that is unusually query-targeted, or answers that changed after a new source was admitted.

Controls That Protect the Knowledge Base

There is no single fix for knowledge-base poisoning. The defense is layered, and each layer answers a different question at a different point in the pipeline.

The order matters. Constrain who can write, record where content came from, review or quarantine incoming documents, then test whether the assembled system holds up.

Provenance. Track where every document came from, who authored it, and when it was added. If you cannot answer those three questions for a document, you should not trust it. Provenance gives you the ability to audit, and auditability is what turns a vague suspicion into a concrete investigation.

Access control. Enforce permissions at the retrieval layer, not by trusting the LLM to decide what is safe. The model has no idea which documents a given user is allowed to see. Access decisions belong in the retrieval pipeline, where you can filter documents based on the user's identity and role before they ever reach the context window.

Content review. Scan incoming documents before indexing. This can range from regex pattern matching for known attack signatures to classification models that flag suspicious content. Review is not a one-time step. It is a continuous process, because attackers adapt their techniques.

Evaluation. Test your RAG system against known-bad queries. Plant a poisoned document in a staging environment and measure whether it changes the answer. Run red-team exercises that simulate the attacks you are trying to prevent. Evaluation tells you whether your other controls are actually working.

A Risk-Based Decision Rule

The most common mistake is assuming that retrieval quality guarantees answer trustworthiness. A high similarity score means the document matches the query. It says nothing about whether the document is true. Those are different questions, and conflating them is how poisoned content slips through.

The second mistake is treating content filtering as a one-time step. Attackers change their techniques. Your review process needs to change with them.

The third mistake is relying on the LLM to notice or refuse poisoned content. The model cannot distinguish a legitimate document from a crafted one—both look like text to be summarized. Asking the model to police its own evidence is like asking a librarian to fact-check every book on the shelf while reading them all aloud.

Here is the decision rule I use: treat every document as untrusted until provenance, access, and review controls have cleared it. That sounds paranoid for a low-risk internal corpus, and it is—if your knowledge base contains only internal wikis with restricted write access, simpler validation may be enough. But the moment your pipeline ingests external content, accepts shared files, or allows broad write access, the paranoid default becomes the engineering standard.

Evaluation sits outside this clearance step. It is not a gate that documents pass through. It is the test of the assembled system: whether poisoned content, once admitted despite your controls, still changes the answers your users see.

Audit Your Knowledge Base

Start with an audit. For each document in your index, ask three questions: Where did this come from? Who can write to this database? Would a planted document change the answers your system produces?

If you cannot answer the first question, you have a provenance gap. If the answer to the second is "too many people," you have an access problem. If you do not know the answer to the third, you need to run an evaluation.

Grounding gives you evidence. Provenance, access, review, and evaluation decide whether that evidence deserves trust. The model will faithfully cite whatever it retrieves—make sure what it retrieves is worth citing.

The adjacent failure mode is prompt injection, where untrusted instructions enter through the live prompt rather than the source corpus. Both attacks exploit the same underlying weakness: the model cannot distinguish trusted from untrusted content on its own. The difference is where the boundary sits. Knowledge-base poisoning corrupts the well. Prompt injection poisons the cup. Defend both, because an attacker only needs one open path.