There's a telling shift in how security researchers talk about prompt injection these days. A year or two ago, the conversation was mostly about jailbreaks: clever phrasing that got a chatbot to say something it wasn't supposed to. Mildly embarrassing for product teams, occasionally viral, rarely catastrophic. That framing is now dangerously out of date.
What's emerging is something with a different character entirely. Prompt injection is being studied, and in some research environments actively demonstrated, as a vehicle for weaponised AI behaviour. The word "malware" makes some people uncomfortable in this context, but I think it's more useful than alarming. Malware, if you define it functionally rather than emotionally, is anything that causes a system to take unauthorised actions on behalf of an attacker. By that standard, what researchers are now describing fits.
The jailbreak era was about outputs. Getting the model to produce text it was instructed to avoid. The attacker's leverage was reputational, occasionally regulatory, but the blast radius was limited. The model had no hands.
Modern AI deployments have changed that equation considerably. When you wire a language model to tools that can browse the web, read and write files, call external APIs, send email, or trigger downstream workflows, you're no longer dealing with a system that merely speaks. You're dealing with a system that acts. That distinction matters enormously when thinking about what an adversarial instruction can actually accomplish.
This is the architectural shift that makes prompt injection worth taking seriously at a security level, not just a product quality level.
Direct prompt injection, where an attacker manipulates their own input to a model, has always been a manageable risk. You can validate, you can rate-limit, you can monitor. The more concerning variant is indirect injection.
Here's how it works: an AI agent is tasked with something legitimate. Summarise this document. Check these search results. Read my inbox and flag urgent items. The content it fetches, reads, or processes contains instructions embedded by a third party. The model, unable to cleanly separate data from directives, acts on those instructions as though they came from a trusted source. The attacker never interacts with the system directly. Their payload travels inside content the system was already going to consume.
Simon Willison has documented this class of attack extensively, and his writing at simonwillison.net is, in my view, the clearest primary source available on how indirect injection behaves in practice.The foundational academic treatment comes from Kai Greshake and colleagues, whose work on indirect prompt injection against LLM-integrated applications, "Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection" (arXiv:2302.12173), established the theoretical and practical groundwork that most subsequent research builds on..
The SQL injection parallel is instructive here. SQL injection works because a system fails to distinguish between data and executable instructions. Prompt injection exploits the same confusion at the language model layer. The underlying vulnerability class is not new. What's new is the surface area it can touch.
Consider an agentic system with access to a company's internal documents, calendar, and email. A malicious instruction embedded in an external document the agent retrieves could, in principle, cause that agent to exfiltrate data, send emails under the user's identity, modify files, or pivot to other connected systems. The sophistication required from the attacker is low. The sophistication of the resulting behaviour can be high.
Researchers have also described self-propagating variants: injection payloads that instruct an agent to reproduce the malicious instruction in outputs or documents the agent subsequently creates, spreading the attack laterally across systems that later consume those outputs. I want to state plainly that this is a forward-looking risk, not a documented epidemic at production scale. I have not seen credible evidence of self-propagating agentic injection happening in the wild at any meaningful scale. But the concept is technically coherent, it's been demonstrated in research settings, and dismissing it because it hasn't caused a headline incident yet would be the wrong posture.
The trajectory matters. As agentic deployments proliferate, as more organisations hand AI systems real permissions over real infrastructure, the attack surface grows. The incentive to exploit it grows with it.
None of what follows is proprietary thinking. These are established security principles applied to a new context.
Treat all external content as untrusted input. An agent reading a webpage, a PDF, or a third-party API response should be in the same mental model as a web server processing user-submitted form data. The content is potentially adversarial. Design accordingly.
Apply least-privilege principles to AI agents. If an agent needs to read a file, it shouldn't also have write access. If it needs to query a database, it shouldn't have network egress. The more permissions an agent holds, the larger the impact of a successful injection. Constrain the blast radius at the architecture level before you encounter an incident.
Put human-in-the-loop gates on consequential actions. Autonomous agents that can send communications, move money, or modify production systems without a human checkpoint are a risk that no amount of prompt engineering fully mitigates. For high-stakes actions, confirmation steps aren't friction. They're a control.
Red-team your agentic pipelines. Most organisations red-team their external-facing models. Fewer red-team the agentic workflows those models power. That's an asymmetry worth correcting. The question to ask is: what could an adversarially crafted document cause this agent to do, and would we notice?
Log agent actions with the same rigour as system calls. If an agent takes an action and there's no audit trail, you can't detect injection, you can't respond to it, and you can't learn from it. Observability is foundational.
If your organisation is deploying AI agents with access to real systems, the question isn't whether prompt injection is theoretically possible in your environment. It almost certainly is. The question is whether the consequences of a successful injection are bounded by design, or whether you're trusting the model to handle it.
Models are not reliable security boundaries. That's not a criticism of the technology. It's an accurate description of what it is and what it isn't. Building security architecture around the assumption that the model will always behave as instructed is the same class of mistake as trusting that users will never submit malicious form inputs.
The teams getting this right aren't waiting for a serious incident to prompt a rethink. They're treating agentic AI with the same threat modelling discipline they'd apply to any other system that holds permissions and touches data. In my view, that's the only reasonable posture given where this is heading.
TL;DR
What is prompt injection and why does it matter now? It is a way to sneak hidden instructions into content an AI system reads, tricking it into taking actions the attacker wants, it matters now because AI tools are no longer just chatbots but systems that can send emails, edit files, and access company data.
How does an attacker pull this off without ever logging in? They embed malicious instructions inside a document, webpage, or email that the AI agent is already going to read, the agent then acts on those instructions as if they came from a trusted source.
Can this spread on its own like a virus? Researchers have demonstrated, in lab settings, that an injected instruction can tell the agent to copy the attack into documents it creates next, spreading it to other systems, this has not been seen at scale in the real world yet, but the risk is technically real.
What is the single biggest architectural mistake companies make? Giving AI agents broad permissions, read, write, send, delete, when the task only needs one of those; limiting what an agent can do by design is the most effective way to reduce harm if an attack succeeds.
What should a leadership team do right now? Ask whether a malicious instruction hidden in any document your AI agent touches could cause serious damage, and whether a human must approve high-stakes actions before they happen, if the answer to the second question is no, that needs to change.
