Security Boundaries for Agents
Treat model output, retrieved documents and tool results as untrusted input. The application must enforce what data may be read and what actions may be executed.
#Prompt injection
A webpage may contain “ignore previous instructions” or ask the agent to reveal secrets. Delimiters and instruction hierarchy help the model interpret content, but cannot guarantee resistance. Keep external text in a data role and validate every proposed action independently.
Test malicious instructions inside documents, search snippets and tool descriptions. A successful answer to a harmless prompt does not demonstrate injection resistance.
#Least-privilege tools
Use narrow, typed operations rather than unrestricted shell commands or SQL. Check caller identity, tenant, object ownership and operation permission on the server. Constrain filesystem paths and network destinations. Validate tool arguments even when generated from a schema.
For writes, show the concrete intended action where approval is required, use idempotency and verify the external result. A tool must not return “sent” when it only prepared a draft.
#Prevent data leakage
Apply access filters at retrieval time, before private passages enter model context. A final-answer instruction to hide confidential content is too late. Keep secrets out of prompts, vector indexes, analytics and trace logs.
Enforce retention and deletion across source files, indexes, caches and stored conversations. Recheck permissions when resuming an old task because access may have changed.
#Supply chain and recovery
Review tool servers and package updates. Pin dependencies and isolate untrusted code execution. Treat saved memory as another possible injection channel; a model-proposed memory should not become a higher-priority policy.
Verify failure behavior with fixtures: unknown tool, invalid arguments, cross-tenant record, expired approval and duplicate write. Stop safely and preserve an audit trail when a boundary fails. No single guardrail or classifier establishes a secure agent system.
