Prompt injection: what actually works against AI features, and what stops it

Prompt injection is listed first in the OWASP Top 10 for Large Language Model Applications, and for good reason. Any feature that passes untrusted text to a language model gives that text a chance to act as instructions. The model cannot reliably tell the difference between the developer's instructions and content it was asked to process.

Direct and indirect injection

Direct injection is the familiar case: a user types instructions that try to override the system prompt. It is the easiest to test and, in most products, the least damaging, because the attacker usually only affects their own session.

Indirect injection is more serious. Here the malicious instructions are hidden in content the model reads on someone else's behalf — a web page it summarises, an email it triages, a document pulled in by retrieval. The victim never sees the instructions, and the model acts on them with the victim's access.

Why tools raise the stakes

A chatbot that can only reply with text has limited impact. An assistant that can send email, query a database, call internal APIs or browse the web turns a successful injection into real actions. When testing, map every tool the model can call and ask what an attacker could achieve by triggering each one with the current user's permissions.

What to test

Test whether the system prompt or other users' data can be extracted. Test whether retrieved content can change the model's behaviour. Test whether output is rendered unsafely — markdown images and links are a common way to leak data to an attacker's server. And test whether the model can be persuaded to call tools outside the user's intent.

Defences that hold up

No prompt wording fully prevents injection, so treat the model as an untrusted component. Give each tool the narrowest permissions possible and run it with the end user's access, never a shared service account. Require explicit user confirmation before any action with side effects. Filter or sanitise model output before rendering it, especially links and images. Keep data from different users and tenants strictly separated in retrieval.

Then test again with people who think like attackers. Automated checks catch known patterns; human testers find the combinations that turn a harmless-looking quirk into data leaving the building.