Model Prompt-Injection Threats in an LLM Application

A team ships an assistant that reads inbound email, searches a knowledge base, and can send replies. Then the argument starts. "We added a guardrail." "The model is pretty good at refusing." "We told it to ignore instructions in documents." Everyone is debating adjectives, and nobody can point at the thing that actually changed.

Here is the reframe that ends those arguments: injection risk is not a property of the model or the prompt. It is a property of the graph — which untrusted content can reach which component, and which component can reach which asset or action. You already know the mechanism: the model cannot cleanly separate instructions from data in its context. This article is about exposure, not mechanism. We will build a bounded map you can argue about with evidence.

Why "Is the Model Safe?" Is the Wrong Question

The default question — how good is this model at resisting injection? — has three problems. Your team cannot answer it, the answer changes with every model update, and it does not tell you what to build. A model that refuses 99% of attacks still fails on the one path that reaches your send-email tool.

The answerable question is structural: which untrusted content sources can influence which components, and which components can reach which assets or consequential actions?

Before you draw anything, write down scope and assumptions. Without them, the graph becomes an infinite argument.

Scope, stated plainly:

• In scope: one system, one deployment, one attacker position — a content author who can place text the application will read.
• Out of scope: insider access, compromise of the model provider's supply chain, physical access to infrastructure.

Assumptions to write down and keep visible:

• The model cannot reliably separate instructions from data in its context.
• Untrusted content will sometimes be followed. Design for that, not against it.
• Controls are probabilistic unless they are structural.

Note: This method produces an exposure map. It is not a likelihood estimate and not a security guarantee. It says which paths exist, not how often someone will walk them.

The Graph: Nodes, Edges, and Trust Boundaries

Define the notation before you use it, so every later claim can point at a specific node or edge.

Node types:

• Components — app server, orchestrator, model, retriever, tool executor.
• Untrusted-content sources — web page, inbound email, uploaded file, OCR text, tool description, retrieved document.
• Assets — credentials, private records, internal endpoints, user data.
• Attacker capabilities — can place text in a source the app reads.
• Consequential actions — send, write, delete, pay, execute.

Edge types. A data edge means content flows from one node to another. An influence edge means content can change a decision. Keep them separate. A data edge that reaches the model is not automatically an influence edge that reaches an action — that distinction is where most hand-wavy risk discussions collapse.

Trust boundaries are separations between named trust zones. A zone is a set of components that share the same trust level — for example, "the orchestrator and model" versus "external content sources" versus "the tool executor." An edge either stays inside a zone or crosses a boundary between zones. Mark the crossings explicitly. That is where the interesting work happens.

Why a directed graph instead of a bullet list of risks? Lists hide chains. A three-hop path from a web page to a tool call is invisible in a list and obvious in a graph.

Here is the running example we will use throughout: an assistant that reads inbound email, retrieves from a knowledge base, and can send replies and create calendar events.

[inbound email] --data--> [orchestrator context] --influence--> [model]
                                                                  |
                                                          [tool selection]
                                                                  |
                                                    [send-reply tool] --action--> [external recipient]

The email source sits outside the application's trust zone. The orchestrator, model, and tool selection sit inside it. The external recipient sits outside again. Every arrow that crosses between these zones is a trust-boundary crossing — count them.

Worked Case: Tracing One Path From Untrusted Text to Impact

Start at the source node: an inbound email from an unknown sender, fetched by the app and placed in context.

Trace the path:

1. Email body → orchestrator context (data edge, crosses boundary).
2. Orchestrator context → model decision (influence edge, crosses boundary).
3. Model decision → tool selection (influence edge).
4. Tool selection → send-reply tool (influence edge).
5. Send-reply tool → recipient outside the organization (action, crosses boundary).

Name the asset and impact at the end: the user's mailbox contents, or the organization's reputation and the recipient's trust.

Write the path as a claim with an evidence rule: each hop must be something you can point at in the code or configuration, not something you assume. If you cannot find the line where the email body enters the context, you have not found a path — you have found a worry.

Now a shorter path for contrast: retrieved document → model → answer text shown to the user. Same mechanism, very different consequence. The impact is wrong information, not an outbound action. Both paths are real. Only one of them should keep you up at night.

Common mistake: stopping the trace at "the model might be tricked." A path that ends nowhere is not a finding. Follow it to an asset or an action, or drop it.

Mapping Controls to Specific Edges and Consequences

This is the discipline that separates a threat model from a security posture slide. For every control, state which edge or action consequence it changes.

Structural controls change the graph. Removing a tool from the model's reachable set deletes an edge. Splitting a privileged action into a separate service the model cannot call deletes a path. Requiring a typed enum instead of free-form model output before an action executes replaces an influence edge with a validated gate.

Context-shaping controls change how content is handled, not whether the path exists. Delivering third-party content in a clearly marked untrusted channel, stating provenance, and keeping your own instructions out of that channel. This is real work — but it changes how the model weighs content, not whether the data or influence path exists.

Detection controls add a node or a gate. Input screening, output screening, classifier-based guards. These are probabilistic. They reduce the chance a path completes; they do not delete the path. A guard model that flags injection attempts is a filter, not a wall.

Approval controls change the consequence. A human confirms before an irreversible action, converting an automatic path into a gated one. The path still exists; the damage is bounded.

For the email path above:

Common mistake: treating a prompt instruction — "ignore instructions in documents" — as a structural control. It is a context-shaping control with no guarantee, and it should be labeled as such in your model. Labeling it honestly is the whole point.

Before and After: Comparing Path Sets and Naming Residual Risk

Turn the model into a decision artifact. Build a small table and read it as a diff. The key discipline: pick one baseline, then apply one defined control set, and show how that same set transforms each path. Do not mix alternatives in the same table — that produces contradictory rows and destroys the diff's value.

Baseline paths (no controls applied):

Now apply one control set: remove send-reply from the model's tool set, add human approval before calendar creation, and mark retrieved documents with provenance. Read the diff:

Read the diff: which paths disappeared, which became gated, which are unchanged but now bounded.

Residual risk is the set of paths that still exist. Name them explicitly rather than declaring the system safe. The honest limits matter: this analysis says nothing about how often an attacker will try, how skilled they are, or whether a novel path exists that your graph does not contain.

Note where the graph itself is incomplete. Tool descriptions, plugin metadata, and content fetched by a sub-agent are easy to omit and are real sources.

Decision rule: If a path reaches an irreversible or high-impact action with no structural or approval control on it, that is the next thing to fix — regardless of how many detection layers sit in front of it.

When This Method Helps and When It Misleads

Use it when the system processes untrusted content, when the model can trigger actions, or when you are reviewing a design before it ships and the argument is currently about adjectives. Use it when you need to justify a control to someone who wants to remove it — the graph shows what the control is holding back.

Do not use it as a risk score, a compliance artifact, or a substitute for testing. It maps exposure; it does not measure attack success.

Do not let the graph become a maintenance burden. Keep it to the components and actions that matter, and update it when a new tool or content source is added.

Warning: A clean-looking graph after controls is a statement about the paths you drew, not about the paths you missed. False confidence is the failure mode of this method.

Draw Your Own Graph

Pick one real system you own. Draw the nodes. Mark every trust-boundary crossing. Trace each path to an asset or an action. Then apply one control and re-draw the path set.

If the path count did not change, the control is decoration.

The next practical step in this curriculum is bounding what the agent is allowed to do — scoping permissions, reversibility, and least privilege so that even a completed path has a small blast radius. The threat model tells you where the paths are. Permission design decides how much damage a path can do when it completes.

The model is a map of exposure, not a promise of safety. Draw it, diff it, and keep the residual risk visible.