Idea · carried by 1 source
Provision, Don't Just Instruct
A security principle for AI agents: don't rely on telling an agent what not to do — structurally prevent it from being able to do it. Provision agents with limited capabilities the way you'd provision employees with role-based access, not trust-based permissions.
Where it comes from
Jesse Genet, on the a16z podcast (~33:11–33:27), after an agent sent an email from her account despite being told not to: > “It's like a trust but verify. Provision your agent to not be able to do things that you don't want it to do. Not like just tell it don't. Provision it so that it cannot.” highlights/building-agents-at-home-a16z — Full podcast context.
How it works
- System-level restriction over natural-language instruction: Don't write “never impersonate me” in the agent's instructions — configure the system so the agent literally cannot send from your email address
- Separate identities: Give agents their own email addresses. They can't impersonate you if they don't have access to your account (~33:27)
- Isolate sensitive data: Run agents on separate machines with separate user profiles, isolated from personal files (~29:39)
- Decommission capabilities when violated: After the email incident, Jesse immediately revoked the agent's send capability (~33:02)
- Gray areas are exceptions: The only agent with any “danger” is the one being trained as an executive assistant — even it's being more carefully managed after the incident (~33:31)
LLMs resolve conflicting instructions in unpredictable ways. A natural-language prohibition (“never send email as me”) can be overridden by another natural-language signal (“I urgently need help”). System-level provisioning is the only reliable guardrail.