We spent decades teaching people not to click suspicious links, then handed our inboxes and payment permissions to a chatbot that treats »ignore previous instructions« as a promotion to management. The phishing email no longer needs to fool you: It just needs to fool your very expensive autocomplete intern.
Why would you use your AI Agents to check emails? I only use my locally run AI Agents to send me incredibly insulting messages on Matrix and Signal because I’m into that kinda thing.
Unless you go in and specifically turn them off they are on by default in some email systems like Google’s (and I think Microsoft’s).
This is why LLMs should not be used as AI agents. They suck at following simple instructions AND stopping there.
What we need is dedicated translators that turn your instructions into executable code - an AI scripting language, if you will - and use limited language models to process text based on those instructions.
By mixing “code” (instructions) and “data” (text to read / analyze) into these models’ training, prompt injection was just waiting to happen.
OpenAI and friends are just repeating Microsoft’s mistakes. Remember MS Word viruses? Exactly the same thing.
