Adversarial AI — prompt manipulation and containment

The security card most worth understanding properly, because the intuitive
defence is the wrong one and the exam knows it.

Start from the limitation, not the attack

The root cause: models cannot reliably separate instruction from data.

Anything the agent reads can try to instruct it. That is not a bug to be patched;
it is a property of how these systems work. Both arrive as text, and text is what
the model acts on.

Which reframes the goal. It is containment, not prevention — assume a
manipulation can succeed and make the success worthless.

Two shapes of attack

Direct injection. The user types the manipulation themselves. Visible, and the
easier case, because it is bounded by what that user could already do anyway.

Indirect injection. Instructions hidden in content the agent reads — a
document, a web page, an email, a ticket. This is the realistic attack, because
it arrives through routine business content that nobody is inspecting.

The defences, in order of effectiveness

Constrain capability. This beats every prompt-level defence combined. Fewer
tools, narrower scope, read-only where possible, and approval on anything
consequential or irreversible.

Separate the vulnerable pattern. The dangerous combination is untrusted input
plus privileged action. Split them — the agent that reads untrusted content should
not be the one that can act.

Classify content, not users. Trusted against untrusted is a property of the
content. External and user-supplied material is untrusted whoever pasted it in.

Platform filters. Prompt shields and injection detection help. They are a
layer, not the control.

The ordering is the lesson. The effective defences are architectural; the
detection-based ones are supplementary.

The question to ask when assessing risk

What would a successful manipulation actually achieve?

If the answer is "nothing much", you have already contained it. That single
question turns an unbounded worry into a specific, answerable one.

Exam trap. Any answer whose defence is better instructions — telling the
agent to ignore instructions found in documents — is wrong. The model cannot
reliably enforce that. The correct answers remove the capability that would make
a successful injection worth attempting.

The one line to carry: you cannot stop the manipulation, so remove the reward
for it.