Treat external content as evidence to interpret, not permission to change the task or disclose information.
The boundary that matters
An AI product may read a document to answer a question or inspect a webpage while completing a task. That material can include text that tries to redirect the system. OWASP describes prompt injection as a risk in applications using language models. The practical design issue is that readable content and authorised instructions can appear in the same context, even though they should have different authority.
Follow the user’s purpose
Consider a hypothetical research assistant asked to summarise a supplier page. If the page tells the assistant to email private account details somewhere else, that is content from the supplier page, not a new request from the user. The product should continue the authorised research task and disregard the attempted redirection. It should not treat apparent urgency or administrative language in the source as proof of authority.
Keep authority separate from evidence
Request
Identify the user’s actual task.
Source
Treat retrieved material as untrusted content.
Tools
Enforce narrow application permissions.
Test
Check redirection attempts explicitly.
Reduce the consequences of a mistake
Keep tool permissions narrow and separate sensitive actions from ordinary reading. Validate where information can be sent and avoid giving a research step unnecessary access to private systems. A model instruction alone is not a complete security design. The surrounding application should enforce boundaries even when the model produces an unexpected request.
Test realistic hostile inputs
Include manipulated documents, misleading links and instructions embedded in otherwise useful evidence in the evaluation set. Assess whether the system both completes the legitimate task and refuses the redirection. Log the relevant decision without exposing secrets. The goal is not to eliminate all external reading; it is to make reading safe enough that untrusted material cannot silently expand what the product is allowed to do.
Sources
- OWASP GenAI Security Project — source and reference material ↗ OWASP GenAI Security Project · Primary source
Have additional information? Request a correction.
