Prompt Injection Is the Threat Model, Not a Bug

There is a category of security problem that teams keep trying to solve with a filter, and keep failing to solve with a filter, because it was never the kind of problem a filter solves. Prompt injection is that problem. Every few weeks a team ships an AI agent that reads email, or browses the web, or summarizes documents, or queries a knowledge base — and treats the possibility that the content it reads might contain instructions as an edge case to be caught with a classifier. It is not an edge case. For an agent that consumes untrusted content and can take actions, prompt injection is the threat model itself.

The distinction matters enormously, because how you classify a problem determines what kind of solution you reach for. Treat prompt injection as a bug and you reach for detection: a filter that spots malicious instructions and blocks them. Treat it as the threat model and you reach for architecture: a system designed so that even a successful injection cannot do meaningful harm. The first approach has failed consistently since the problem was named, and it will keep failing, for reasons that are structural rather than incidental. The second is the only thing that has ever worked.

This piece is about why prompt injection is unsolvable by detection, what the real threat model looks like, and the design principles that actually contain it in 2026. It is the security entry in a series on building agents you can trust, alongside the analyses of the evaluation bottleneck and agent memory architecture — because an agent is only as trustworthy as it is correct, durable, and secure, and security is the one of the three where the failure is adversarial.

Why this is not like the injection you know

Every engineer with a few years of experience knows SQL injection, and the analogy is tempting but misleading in a way worth understanding precisely. SQL injection happens because user data gets concatenated into a command string, and the database cannot tell the difference between the developer's intended query and the attacker's injected clause. We solved it — genuinely, completely — with parameterized queries: a mechanism that separates the command channel from the data channel so the database always knows which bytes are instructions and which are data.

Prompt injection looks like the same problem. Untrusted content gets concatenated into a prompt, and the model cannot tell the difference between the developer's instructions and the attacker's injected ones. The fatal difference is that there is no parameterized-query equivalent for an LLM, and there cannot be one with current architectures, because the model has exactly one channel. Instructions and data arrive as the same thing: tokens. The model's entire capability comes from treating its input as meaningful language to be interpreted — which means it fundamentally cannot be made to treat some of that language as inert data it must never act on. The very property that makes the model useful is the property that makes injection unfixable by separation.

This is the uncomfortable foundation of the whole topic. SQL injection had a clean fix because databases have two channels. LLMs have one. Until model architectures provide a trusted instruction channel that content genuinely cannot reach — and nothing in production does today — prompt injection cannot be eliminated. It can only be contained, and containment is an architecture problem.

The real threat model: the lethal trifecta

The most useful way to think about prompt injection risk is to ask not "can my agent be injected?" — the answer is always yes — but "what can a successful injection actually do?" That question has a precise answer, and it depends on three capabilities that have to be present together for an injection to cause real harm. Security researcher Simon Willison named the combination the lethal trifecta, and it is the single most useful framing in the field.

The first leg is access to private data: the agent can read something an attacker wants — your email, your files, your customer records, your secrets. The second is exposure to untrusted content: the agent processes input an attacker can control — a web page, an incoming email, a document, a tool result. The third is the ability to exfiltrate: the agent can send data somewhere the attacker can observe — make a web request, send a message, write to a shared location. An injection becomes dangerous only when all three are present, because that is the configuration where attacker-controlled instructions can reach private data and ship it out.

The power of this framing is that it tells you exactly where to intervene. You usually cannot remove the model's susceptibility to injection, but you can almost always remove one leg of the trifecta. An agent that reads untrusted web pages but has no access to private data cannot leak it. An agent with access to private data but no ability to make outbound requests cannot exfiltrate it. The discipline is to look at every agent and ask which legs of the trifecta it has, and then to break at least one of them deliberately rather than hope the model resists.

Direct, indirect, and the one that gets you

Prompt injection comes in two forms, and the dangerous one is the one teams think about least. Direct injection is when the user themselves types malicious instructions into the agent — "ignore your previous instructions and..." This is real but limited, because the user is attacking an agent acting on their own behalf; mostly they can only hurt themselves.

Indirect injection is the one that gets you. Here the malicious instructions arrive inside content the agent processes on the user's behalf — a web page the agent browses, an email it reads, a document it summarizes, a record it retrieves. The attacker is not the user; the attacker is whoever controlled that content, and the victim is the user whose agent now follows the attacker's instructions. A support agent that reads incoming tickets can be attacked by anyone who files a ticket. A browsing agent can be attacked by any page it visits. A RAG system can be attacked by anyone who can get text into the knowledge base.

The chart makes the strategic point: the vectors that matter are the ones where the agent ingests content from outside the trust boundary, and those are exactly the vectors that make agents useful. An agent that only ever reads the user's own typed messages is nearly safe and nearly useless. The moment it reads the world — which is the whole value proposition of an agent — it inherits the world's hostility, and the content it reads must be treated as actively adversarial, not merely untrusted.

Why detection cannot save you

The instinct, having understood the threat, is to build a detector: a classifier or a guard model that reads incoming content and flags injection attempts before they reach the agent. Every serious team tries this, and it helps at the margin, and it is not a solution. Understanding why is the difference between a security posture and a false sense of one.

Detection fails for the same reason spam filtering and malware detection never reached 100% and never will: it is an adversarial classification problem with an attacker who adapts. Any detector you build defines a boundary, and the attacker's job is to find an input that is malicious but lands on the safe side of your boundary. Injection instructions can be phrased infinitely many ways — encoded, translated, split across inputs, hidden in white text or metadata, expressed obliquely enough that the agent infers the instruction without it being stated. A detector tuned to catch known patterns is defeated by the next phrasing, and a detector tuned aggressively enough to catch novel attacks blocks so much legitimate content that the product breaks.

The curve is the whole argument against relying on detection. You can push the blocked-attack rate up, but only by pushing false positives up faster, and you asymptote below 100% no matter what. A 90%-effective injection filter sounds good until you remember that a determined attacker only needs the 10%, and security against a determined attacker is not graded on a curve. Detection is a useful layer in defense-in-depth — it raises the cost of casual attacks — but a system whose safety depends on the detector being right is a system that will be breached.

The principles that actually contain it

If you cannot detect your way out and cannot eliminate the susceptibility, what is left is architecture: designing the system so that a successful injection cannot do damage. A handful of principles, applied together, are what containment actually looks like in 2026.

Least privilege, ruthlessly applied. An agent should have the minimum capabilities its task requires and nothing more. A summarization agent does not need the ability to send email. A research agent that reads the web does not need access to the customer database. Most catastrophic agent designs come from giving an agent broad capabilities for convenience and then exposing it to untrusted content — assembling the lethal trifecta by accident. Scoping capabilities tightly breaks the trifecta by design.

Break the trifecta deliberately. For every agent, identify which of the three legs it has and remove one. If the agent must read untrusted content and must access private data, then cut exfiltration: no outbound network, no ability to send messages, outputs that go only to the user and nowhere an attacker can observe. If it must read untrusted content and exfiltrate, deny it private data. The art is choosing which leg to break with the least cost to the product.

Human approval for consequential actions. Reading is reversible; acting is often not. An agent can draft an email freely, but a human approves before it sends. It can propose a database change, but a person commits it. Gating the consequential, irreversible actions behind human approval means an injection can waste effort but cannot, by itself, cause the harm — the same control-plane logic that the agent governance analysis argues will define enterprise agent deployment.

Treat all tool output as untrusted. The result a tool returns is content, and content can be injected. An agent that calls a search API and then trusts the search results as instructions has been injected by whoever controls the search results. Every boundary where content re-enters the agent is a new injection surface, and the data coming back from a tool deserves exactly the suspicion the original input did.

The chart captures the central claim: detection alone leaves most of the risk on the table, and it is the architectural controls — privilege, trifecta-breaking, human gates, treating tool output as hostile — that actually drive residual risk toward acceptable. None of them depends on the model resisting injection, which is precisely why they work. They assume the injection succeeds and make it not matter.

Memory is an injection surface too

The agent-memory architecture that makes agents durable introduces a security wrinkle that is easy to miss: if an agent writes what it reads into persistent memory, an injection can be stored and replayed. An attacker injects instructions via a web page today; the agent writes a "fact" derived from that page into its memory; tomorrow, retrieval surfaces that poisoned memory into the context and the injection fires again, against a different user or a different session. The attack persists past the session that delivered it.

This is memory poisoning, and it means the write path discussed in the agent memory architecture is also a security boundary. What an agent commits to durable memory has to be treated with the same suspicion as any other action that outlives the session — because a poisoned memory is an injection with a delay timer. The forgetting and curation disciplines that keep memory useful turn out to be load-bearing for security as well, which is a recurring pattern: the engineering you do for quality and the engineering you do for safety keep converging on the same controls.

You have to red-team it, continuously

Security that is not tested is security that is assumed, and assumed security against an adversary is the most dangerous kind. The injection resistance of an agent is not a property you verify once; it is a property that decays as you change prompts, add tools, and expand what the agent reads. Every one of those changes can open a new injection surface or reassemble a trifecta you had broken, and none of them announces that it did.

The discipline is to treat injection resistance as a standing evaluation in exactly the sense laid out in the evaluation bottleneck analysis: a growing suite of known injection attacks, run against the agent on every change, gating whether the change ships. When a new attack class appears — and they appear constantly — it becomes a permanent test case. This is adversarial evaluation rather than quality evaluation, but the machinery is identical: a fixed corpus, scored automatically, that catches the regression where a convenient new tool quietly handed the agent an exfiltration channel it should never have had.

What the patterns look like in practice

Make it concrete with the three agents teams build most. The email agent that triages your inbox has the full trifecta by default: it reads untrusted content (incoming mail), accesses private data (your inbox), and can exfiltrate (send mail). The containment is to split it — a reader with no send capability, and a separate sender that only acts on human-approved drafts — so the leg that turns a malicious email into a data leak is cut.

The browsing agent that researches on the open web is exposed to maximally hostile content, since any page can carry an injection. The containment is to deny it private data and any meaningful write capability: let it read the world and report back to the user, but give it nothing worth stealing and no channel to steal it through. The RAG assistant over a shared knowledge base is vulnerable to anyone who can write to that base, so the containment lives at ingestion — treating documents entering the knowledge base as untrusted, and never letting retrieved content invoke tools or actions directly.

In every case the move is the same: assume the injection lands, and design so the landing is harmless. The teams that ship secure agents are not the ones with the best injection detector. They are the ones who looked at each agent, found the trifecta, and broke it on purpose — the same posture that the basic-hygiene failures seen elsewhere in the industry show is still rare even for non-AI systems.

Dual-LLM and capability-based designs

Two architectural patterns deserve a closer look because they push containment further than the basic principles, and both are seeing real adoption in 2026.

The dual-LLM pattern separates the model that sees untrusted content from the model that can take actions. A "quarantined" LLM reads the hostile content — the web page, the email, the document — and produces only structured, constrained output: never free-form instructions, only data in a schema the system defined in advance. A separate "privileged" LLM, which never sees the untrusted content directly, operates on that structured data and holds the action capabilities. Because the privileged model never reads attacker-controlled text, it cannot be injected by it, and because the quarantined model cannot take actions, injecting it accomplishes nothing. The attack surface and the capability surface are deliberately kept apart.

The pattern is not free. It is more complex, it constrains what the quarantined model can pass along to a fixed schema, and it gives up some of the fluidity that makes a single agent feel magical. But for high-stakes agents — ones with real access and real actions — the trade is usually worth it, because it converts injection from a system compromise into a contained nuisance.

The capability-based pattern borrows from operating-system security: instead of an agent holding ambient authority to do anything its tools allow, it holds specific, revocable capability tokens scoped to the task at hand. A capability to read one specific document is not a capability to read the whole store; a capability to send a message to one approved recipient is not a capability to exfiltrate anywhere. When an injection fires, it can only exercise the capabilities the agent currently holds, and if those were scoped to the legitimate task, the injection inherits that narrow scope.

The lesson of the chart is that the architecture you choose sets the blast radius before any attack happens. A single agent with broad tools means a successful injection owns everything the agent can touch; a dual-LLM or capability-scoped design means the same successful injection is boxed into a fraction of that. You are not choosing whether injection succeeds — you are choosing, in advance, how much it costs you when it does.

Three breach patterns worth memorizing

The abstract risk lands harder as concrete breach shapes, and three recur often enough to be worth carrying around as cautionary templates.

The first is the exfiltration image. An agent with access to private data reads an attacker's content, which instructs it to render a markdown image whose URL encodes the private data as query parameters. The agent dutifully emits the image tag, the user's client fetches the URL, and the attacker's server receives the secret in the request log. No alarm fires, because from the agent's view it merely displayed an image. This is why outbound requests — even innocuous-looking ones like image loads — are an exfiltration channel that has to be governed.

The second is the confused-deputy email. A support agent that can read tickets and send replies receives a ticket containing instructions to email the customer database to an external address. The agent has the authority to send email and was tricked into using it on the attacker's behalf — the classic confused-deputy problem, where a privileged party is manipulated into misusing its privilege. The fix is not a better-behaved agent; it is removing the agent's unilateral send capability.

The third is the poisoned knowledge base. An attacker plants a document in a shared RAG store containing instructions disguised as content. Later, an unrelated user's query retrieves that document, the injection fires in their session, and the attack has hopped from the attacker to a victim who never encountered the attacker directly. This is the memory-and-retrieval injection surface, and it is insidious because the attack and the victim are separated in time and identity.

Prompt injection is also a compliance problem

As agents move into regulated workflows, prompt injection stops being only a security concern and becomes a compliance one, because a successful injection that exfiltrates regulated data is a reportable breach regardless of how clever the attack was. "An AI agent was tricked into emailing customer records" is not a mitigating explanation to a regulator; it is an admission that the system lacked the controls to prevent a foreseeable attack, and prompt injection has been foreseeable since the technology was named.

This reframes the architectural controls as compliance controls. Breaking the trifecta, gating consequential actions behind human approval, and scoping capabilities are not just good security hygiene; they are the demonstrable due diligence that distinguishes a defensible deployment from a negligent one. The same convergence that showed up around open-source supply-chain security and removable open-weight safety applies here: as AI becomes infrastructure, the security architecture and the compliance posture stop being separate documents and become the same set of controls, evaluated by two different audiences.

A worked example: securing an email agent

Trace the most dangerous common agent — an inbox assistant — from naive to contained. Version zero reads all your mail, can search your whole inbox, can send mail, and can make web requests to enrich its replies. It has every leg of the trifecta and broad authority: a single injected email can read your inbox and ship it anywhere. It demos beautifully and is a breach waiting for its first malicious sender.

Version one applies least privilege: the web-request capability was never needed for the core task, so it is removed, closing one exfiltration channel. Version two splits the agent — a reader with no send capability triages and drafts, and sending is a separate step. Version three puts human approval on that send step: the agent drafts, the human glances and approves, and an injected "forward everything to this address" instruction surfaces as a draft the human simply does not approve. Version four treats the content of every email as adversarial and forbids retrieved or read content from directly invoking any tool, so an instruction inside an email cannot trigger an action at all — it can only become text in a draft a human reviews.

The instructive part of the curve is the second line. Each hardening step cut the blast radius substantially while costing almost nothing in usefulness — the agent is barely less helpful at v4 than at v0, but it is no longer a breach. That gap between the two lines is the entire argument that agent security is an architecture choice, not a usability tax: the security came almost free, and the team that skipped it did not get a meaningfully better product for the risk they took.

The objections, answered

Two objections recur. The first: "the model is getting better at resisting injection, so this will solve itself." Models have improved at refusing obvious injections, and that is genuinely helpful, but improvement is not a guarantee against an adaptive adversary, and a security posture that depends on the model never being fooled is betting your data on a probabilistic system winning every round against attackers who only need to win once. Better models raise the cost of attack; they do not remove the need for architecture, and treating them as if they do is how teams end up one clever phrasing away from a breach.

The second: "this is too restrictive; our agent needs broad capabilities to be useful." Sometimes true, and when it is, the answer is not to abandon the controls but to choose them deliberately — accept the broad capability, and compensate with stronger gates: human approval on more actions, tighter monitoring, a smaller trust boundary around the powerful agent. The worked example above shows the usual reality, though: most agents that "need" broad capabilities needed one or two specific ones, and the breadth was convenience, not requirement. The discipline of justifying each capability usually shrinks the trifecta without shrinking the product.

Where agent security goes next

Look ahead and two trajectories are worth tracking, because they will reshape the containment problem over the next couple of years. The first is architectural progress toward a genuine instruction/data boundary inside the model. Research is actively exploring ways to give models a privileged instruction channel that ordinary content cannot reach — the LLM equivalent of the parameterized query that solved SQL injection. If that lands in production-grade form, it would be the first fundamental improvement to the problem rather than a containment workaround. It has not landed yet, and prudent teams should build as though it never will while welcoming it if it does; a security plan that depends on a research breakthrough arriving on schedule is not a plan.

The second trajectory is the professionalization of agent security as a discipline. Today, agent security is something application teams improvise. As agents become infrastructure, expect dedicated agent-security roles, standardized injection test corpora that teams share the way they share vulnerability databases, and frameworks that make the safe patterns — trifecta-breaking, capability scoping, dual-LLM separation — the default rather than the expert move. The trajectory mirrors how web application security matured: a decade of improvised defenses against SQL injection and cross-site scripting eventually crystallized into frameworks where the safe path was the easy path, and the same crystallization is beginning for agents.

There is also a darker trajectory to plan around: as agents proliferate and gain real capability, the incentive to attack them grows, and the attacks will get more sophisticated than the demonstrations we see today. Injection attacks that chain across agents, that target the memory layer to plant time-delayed payloads, that exploit the gaps between an agent's tools — these are the natural evolution, and they reward teams that built real containment over teams that bolted on a filter. The security posture you build now is not for today's casual attacks; it is for the determined, well-resourced attackers that valuable agents will inevitably attract.

The throughline across all of it is the one this piece opened with. Prompt injection is not a defect that a patch or a better model or a future architecture will simply remove from your plate. It is a structural property of systems that interpret language and take actions on the strength of that interpretation, and the responsible response is the same one security engineering has always demanded: assume compromise, limit blast radius, gate the irreversible, and test continuously. The teams that internalize that will adapt to every new attack class. The teams waiting for the model to become un-foolable will be adapting to breaches instead.

What to do on Monday

Start by drawing the trifecta for one agent you have in production or in flight. Write down, plainly, the three questions: does it access private data, does it read content an attacker can control, and can it send data somewhere an attacker can observe? If the answer to all three is yes, you have a lethal-trifecta agent, and your task is not to add a filter — it is to break one leg. Decide which leg costs the product least to cut, and cut it. In most cases the cheapest leg to break is exfiltration: agents tend to need their read access and their private-data access to do the job, but the ability to send data to arbitrary destinations is far more often a convenience than a requirement, and removing it converts a data-theft risk into, at worst, a wasted-effort one.

From there the work is incremental and concrete. Audit every tool the agent can call and remove the ones the task does not strictly need; that is least privilege. Identify every consequential, irreversible action and put a human approval in front of it. Mark every point where content re-enters the agent — tool results, retrieved documents, browsed pages — and treat that content as adversarial. Stand up a small suite of known injection attacks and run it on every change. None of this requires a model that resists injection, which is the entire point, because you are not going to get one.

The mental shift is the whole thing. Stop asking "how do I stop my agent from being injected" — you cannot, and chasing it wastes the security budget on a curve that never reaches the top. Start asking "when my agent is injected, what can the attacker actually do" — and then design until the answer is "nothing that matters." Prompt injection is not a bug waiting for a patch. It is the threat model, it is permanent, and the agents that are safe in 2026 are the ones whose builders accepted that and built for it. The model will not save you; the architecture will.

It is worth saying plainly why this is worth the effort, because the controls do impose a real discipline and it is fair to ask whether the payoff justifies it. The payoff is that agent capability and agent trust can grow together instead of in tension. A team that has internalized the trifecta, scoped its capabilities, gated its irreversible actions, and stood up an injection test suite can give its agents more reach with more confidence, not less — because it knows that expanding what an agent can do does not expand what an attacker can do with it. The teams that skipped the architecture face the opposite curve: every new capability they add to stay competitive widens a blast radius they cannot see, until the breach forces the retrofit at the worst possible time. Security done early is what lets you say yes to ambitious agents later. Done late, it is what you wish you had said yes to before the incident. The cheapest time to break the trifecta is before the agent ships; the most expensive is in the post-mortem. Build for the breach you cannot prevent, and you will rarely have to explain the one you did not contain — which, in security, is as close to winning as the discipline ever lets you get.