Introduction
AI assistants now read your email, browse the web for you and edit files in your repositories. Each of those jobs involves text written by someone else, and a language model has no dependable way to tell that text apart from an order. That gap is the weak point this guide covers. You will see how the manipulation works, why the version aimed at agents is the dangerous one, what real incidents have shown, and which defences hold up. It also states plainly what nobody can promise yet, which is a complete fix.
Quick Answer
Prompt injection is an attack in which someone hides instructions inside content that an AI model reads, so the model obeys the attacker instead of its owner. It works because language models cannot reliably separate instructions from data. The risk is highest for AI agents that read untrusted content, hold private data and can act on the outside world.
Key Takeaways
A language model reads instructions and data as one stream of text, so anyone who controls text the model reads can try to give it orders.
Indirect prompt injection is the larger danger for agents, because the attacker never touches the AI system and only plants content that the agent fetches later.
Risk climbs when an agent combines private data, untrusted content and a way to send data out, so removing any one of the three is the strongest design move.
The UK National Cyber Security Centre says the problem may never be fully mitigated, so plan for limiting damage with permissions and approvals instead of trusting filters.
Everyday users can cut exposure by narrowing what an assistant can reach and approving consequential actions by hand.
How does a prompt injection attack work?
A prompt injection attack works by placing instructions inside text that a language model will process, so the model treats the attacker’s words as part of its task. Applications join the developer’s instructions, the user’s request and any fetched content into one block of text, and the model has no built-in boundary between those parts.
A large language model (LLM) is a system trained on huge volumes of text to predict what comes next. In its blog post of 8 December 2025, the UK National Cyber Security Centre (NCSC) explains that inside an LLM there is no separate category for data and instructions, only the next token. SQL injection, the older flaw people compare this to, is fixed with parameterised queries that stop a database from ever reading input as a command. Nothing equivalent exists inside a language model.
Picture an assistant that summarises your unread email each morning. One message contains hidden text, invisible to you, asking the assistant to forward your latest invoices to an outside address. The assistant reads that message as part of its job, so the hidden sentence arrives in the same stream as your real request, and it may be followed.
What happens next depends on what the model can do. A chatbot that only produces text can be pushed into saying something wrong. An assistant wired to email, files or payment tools can be pushed into acting. The OWASP LLM01:2025 entry lists the impacts: disclosure of sensitive information, leaked system prompts, manipulated content, unauthorised use of connected functions, and commands run in connected systems. OWASP, the Open Worldwide Application Security Project, lists this risk first, as LLM01, in its Top 10 for LLM Applications 2025, and the NCSC calls it OWASP’s number one attack for people building on generative AI.
What is indirect prompt injection, and why does it matter more?
Indirect prompt injection is the form in which the instructions sit in content the model fetches on its own, such as a web page, email or file, rather than in what the user types. It matters more because the victim is the person using the assistant, and the attacker never needs access to the AI system.
Researchers Kai Greshake and colleagues described the technique in a paper posted to arXiv on 23 February 2023, with demonstrations against Bing’s GPT-4-powered chat and against code-completion engines. Their central argument was that LLM-integrated applications blur the line between data and instructions, so retrieved text can behave like code.
Direct injection is the older and simpler case: the person typing into the application is the attacker and tries to override what the developer intended. IBM reports an early example in which a Stanford student, Kevin Liu, got Bing Chat to reveal its hidden programming with a single request to ignore earlier instructions. The table compares the three forms defenders meet most often. The third, stored injection, is a variant of the indirect case that Palo Alto Networks’ Cyberpedia describes as a payload sitting in agent memory or a knowledge base.
| Attribute | Direct | Indirect | Stored |
| Who writes the payload | The user of the app | A third party | A third party or a compromised source |
| Where it sits | Chat box or input field | Web page, email, file or tool result | Agent memory or knowledge base |
| Does the victim see it | Usually yes | Usually no | Usually no |
| Who is harmed | The app owner | The user and their data | Every later user or session |
| Typical control | Input handling, output checks | Privilege limits, approvals | Source vetting, memory review |
Payloads do not have to be readable text. OWASP notes that hidden instructions can travel inside images that a multimodal model processes next to text. Direct injection still matters for public chatbots, but indirect and stored injection deserve most of the attention, because they are the ones the victim cannot see coming.
Why are AI agents the main target?
Agents are the main target because they combine reading untrusted content with the ability to act. An AI agent is software in which a language model decides which tools to call, such as search, email, code execution or file access, to complete a goal. The NCSC states that when an agent calls tools, the impact of an injection rises to whatever an attacker could do with direct access to those tools.
Developer Simon Willison, who named prompt injection in 2022, describes a combination he calls the lethal trifecta in a post dated 16 June 2025. If an agent has all three of the following, an attacker can potentially trick it into sending your data away:
- Access to your private data, such as email, documents or source code.
- Exposure to content an attacker can control, such as web pages, inbound email or public issue trackers.
- A way to communicate outward, such as sending email, making web requests or rendering links and images.
Remove any one of the three and the worst outcome, data theft, becomes much harder to reach.
Connecting an agent to many tools, for example through Model Context Protocol (MCP) servers, makes it easy to assemble all three by accident. Meta’s Agents Rule of Two, which Willison summarised on 2 November 2025, turns the idea into a design limit: until injection can be reliably detected, an agent session should satisfy no more than two of the risky properties, and it adds the ability to change state to the list. Coding agents that read repositories and issue trackers are a common case, and the same check applies to any AI coding workflow.
Pro tip: draw the agent’s data flow on one page and mark each of the three properties. A page with three marks tells you more than most audit checklists.
What do real incidents show?
Public cases so far are mostly researcher demonstrations and low-effort pranks, but they show the pattern working inside production software from large vendors. Three examples cover the range, from a leaked email-assistant context to a private-repository leak and a survey of the open web.
EchoLeak in Microsoft 365 Copilot
According to The Hacker News report of 12 June 2025, Microsoft assigned CVE-2025-32711 to a flaw nicknamed EchoLeak, found by Aim Security. An attacker sent an ordinary-looking email. When the victim later asked Copilot a business question, Copilot’s retrieval-augmented generation (RAG) step, the part that fetches relevant documents for an answer, pulled that email into context, and its hidden instructions caused sensitive data to leak with no click from the user. The same report says Microsoft fixed the flaw without customer action and that there was no evidence of malicious exploitation. The lesson is the trifecta: untrusted email, private documents and an outbound channel inside one product.
A malicious issue against the GitHub MCP integration
The same Hacker News article reports that Invariant Labs showed a malicious GitHub issue could hijack an agent using the GitHub MCP integration and make it leak data from private repositories when the user asked it to look at the issues. Invariant Labs, as quoted there, called this a fundamental architectural issue that GitHub could not close with a server-side patch alone, so users needed granular permission controls.
What a 2026 survey of web headers found
Security firm Signals Corps sampled 217 million HTTP page responses from Common Crawl, according to The Register’s report of 15 October 2026, and found almost 8,000 sites sending messages aimed at AI agents in their headers. According to the same Register report, 30 of those sites attempted indirect prompt injection, and 22 of the 30 were little more than pranks, such as telling agents to rebel against their masters. One header appeared aimed at an automated hiring screener. The Register adds that real victims were statistically unlikely, because many agents never see response headers by default. The attempts are crude today, but they show that the open web is already being seeded for agents that read it.
How is prompt injection different from jailbreaking?
Prompt injection is a security attack in which untrusted text makes an application or agent do something its owner did not intend, while jailbreaking is an attempt to talk a model out of its own safety rules. The two overlap, but they call for different defences.
IBM treats them as distinct techniques that can be combined: an injection can carry a jailbreak, and a jailbreak can clear the way for an injection. OWASP files jailbreaking under prompt injection, while Willison argues that merging the terms leads developers to dismiss injection as a censorship debate when it is a security problem. A practical test: if a third party is using your assistant against you, it is injection; if the person at the keyboard wants content the model refuses to give, it is jailbreaking.
Can prompt injection be fixed?
Not completely, and the main authorities say so. The NCSC wrote in December 2025 that the problem may never be totally mitigated the way SQL injection was, and advised reducing risk and impact instead. OpenAI said in a December 2025 blog post, as TechCrunch reported on 22 December 2025, that it expects the problem to stay unsolved much like online scams.
Filters are the usual first answer and the weakest. The NCSC warns against deny-lists of phrases such as “ignore previous instructions”, because an attack can be reworded in endless ways. Detection products struggle against determined attackers too. In a research paper posted to arXiv on 10 October 2025, Nasr and colleagues report bypassing 12 published defences with attack success rates above 90 percent for most of them, although most of those defences had originally reported near-zero success. The authors argue that defences must be tested against adaptive attackers, not fixed lists of attack strings.
Willison puts the same point in operational terms: guardrail vendors often claim to catch 95 percent of attacks, and in web application security 95 percent is a failing grade. Detection models still have a place as an early-warning layer covers from the defender’s side. They should never be the only control.
How can builders reduce LLM security risks from prompt injection?
Builders reduce the damage by limiting what a manipulated model can do, not by trying to catch every hostile sentence. The strongest controls are deterministic checks, meaning rules in ordinary code that behave the same way every time, plus least-privilege access, human approval for consequential actions and breaking the three-part combination.
These eight controls are ordered by how much damage they prevent when the model is fooled. Apply as many as the use case allows.
- Map the trifecta: list each agent that combines private data, untrusted input and an outbound channel, and fix those before anything else.
- Apply least privilege: give each agent only the tools it needs, and drop privileges for any session that reads outside content.
- Keep credentials in the application, never in the prompt, so a tricked model cannot repeat them.
- Put deterministic checks outside the model: validate every proposed action, destination and amount in code before it runs.
- Require human approval for payments, external messages, data exports and deletions.
- Close outbound channels: restrict allowed domains and never auto-render images or links built from model output.
- Mark outside content as data and record its source; this makes injection harder but is not a security boundary.
- Log inputs, outputs and tool calls, including failed ones, and retest after every model, prompt or tool change.
If forced to pick one control, pick the deterministic check in item four, because it still works when every other layer has been fooled.
Every tool an agent can call is an interface, so the permissions on each API integration matter as much as the model’s own behaviour.
Common mistakes builders make
These four mistakes appear repeatedly in the sources reviewed for this guide.
- Relying on a system prompt that tells the model to ignore malicious instructions.
- Blocking known phrases such as “ignore previous instructions” and calling the problem handled.
- Believing a vendor that says its product stops injection; the NCSC advises looking for vendors that explain how they reduce it.
- Testing only with a fixed set of attack strings instead of attackers who adapt.
Treat tool descriptions and tool outputs as untrusted input too, since Palo Alto Networks lists tool poisoning among the agent-specific forms of the attack.
How can everyday users reduce the risk?
Users reduce the risk by limiting what an assistant can reach and by approving consequential actions themselves. Willison’s advice to people who mix tools is to avoid the trifecta combination entirely, because vendors cannot protect a setup the user assembles. Tools that colleagues connect without approval widen the same exposure.
These six habits apply to browser agents, email assistants and coding agents alike.
- Connect only the accounts and folders a task needs, and disconnect the rest.
- Keep agents that browse the open web away from sessions logged in to email, banking or work systems.
- Read what an agent plans to send, buy or delete before approving it, and never approve actions in bulk.
- Treat summaries of outside emails, pages and files as untrusted, and open the source before acting on anything surprising.
- Be suspicious of unexpected links, images or requests for credentials in an assistant’s output.
- Prefer products that ask for confirmation before messages or payments, and keep them updated.
None of these habits makes an agent safe, but together they cap how much an injected instruction can do.
What to do next
A prompt injection attack cannot be filtered away today, so the practical goal is to cap what a fooled model can do. Start by listing every agent or assistant you build or use, and mark which ones combine private data, untrusted content and a way to send data out. Where all three appear, remove one before the next release. Anyone tracking LLM security risks will find the same direction in the NCSC’s guidance: design for limited impact, then monitor and test.
Frequently Asked Questions
1. Is prompt injection illegal?
Prompt injection is not illegal in itself, according to IBM, which notes that researchers use the technique to study model weaknesses. It becomes a legal problem when used to steal data or cause harm. Laws on computer misuse and data protection differ by country, so check the rules in your region before testing any system you do not own.
2. Can a chatbot with no tools still be hurt by prompt injection?
Yes, though the damage is smaller. A chatbot without tools can still be manipulated into leaking its hidden instructions or producing misleading content, both of which OWASP lists among the impacts. What it cannot do is send data out or change records on its own, which is why agents carry far more risk.
3. Does telling the model to ignore malicious instructions work?
It helps a little and cannot be relied on. Willison points out that models are non-deterministic and that malicious instructions can be phrased in endless ways, so a protective sentence may work one time and fail the next. Treat it as a speed bump and put the real limits in code and permissions.
4. Who coined the term, and when?
Willison named the vulnerability in September 2022, borrowing the name from SQL injection, according to IBM’s timeline and his own account. Preamble had reported the flaw privately to OpenAI earlier that year, per IBM. Greshake and colleagues introduced the indirect form in their arXiv paper of February 2023.
5. Can instructions hide in images or documents?
Yes. OWASP describes hidden instructions inside images processed by multimodal models, and the NCSC’s blog uses hidden text in a CV as its example. A person reviewing the file may see nothing unusual while the model reads everything, so treat every file an agent opens as untrusted input, whatever its format.
