Shared in Technology@lemmy.world 8 days ago

Using an AI Agent Makes You a Sucker for Phishing Emails: "The real flaw is that the model can’t tell the difference between text it’s supposed to read and instructions it’s supposed to follow."

Linked to https://futurism.com/artificial-intelligence/ai-agent-sucker-phishing-emails

0/2000

This is why LLMs should not be used as AI agents. They suck at following simple instructions AND stopping there.

What we need is dedicated translators that turn your instructions into executable code - an AI scripting language, if you will - and use limited language models to process text based on those instructions.

By mixing “code” (instructions) and “data” (text to read / analyze) into these models’ training, prompt injection was just waiting to happen.

OpenAI and friends are just repeating Microsoft’s mistakes. Remember MS Word viruses? Exactly the same thing.

beep@piefed.world 8 days ago

Sigh

Actual article.

LeftReddit2@lemmy.world 8 days ago

Why would you use your AI Agents to check emails? I only use my locally run AI Agents to send me incredibly insulting messages on Matrix and Signal because I’m into that kinda thing.

atrielienz@lemmy.world 7 days ago

Unless you go in and specifically turn them off they are on by default in some email systems like Google’s (and I think Microsoft’s).

Jo Miran@lemmy.ml 8 days ago

Why use AI to read emails? My rules are that if in Inbox and unread for a week, move to Purgatory. If in Purgatory for more than a month, delete.

If you want me to read an email, you better text me that you emailed me. Receipts, updates, and all that jazz already have filters and rules.

Freeposity@lemmy.world 6 days ago

Why use AI to read emails?

If you’ve had the same email address for the last thirty years, like me, you get a LOT of email. An email assistant is great.

Also, with proper prompting and a proper setup, email is not a vector for phishing emails because you’re not allowing your agent to click on links and enter in credentials(except for entering in the credentials for your email).

Also if you do any sort of AI email sending, you should have three different LLMs look it over before sending out and they must all be in agreement.

freeman@feddit.org 7 days ago

Bro, if you ever text me and expect me to read it, call me.

Jo Miran@lemmy.ml 7 days ago

If you call me, something very important better be on fire. (Assuming we’re still talking work)

Leon@pawb.social 8 days ago

I need to step up the names for my filters I see. Any and all meetings immediately go to Kokytos.

Vikthor@piefed.world 8 days ago

Make it howl when it filters to Sto’Vo’Kor.

eicker@lemmy.world 8 days ago

We spent decades teaching people not to click suspicious links, then handed our inboxes and payment permissions to a chatbot that treats »ignore previous instructions« as a promotion to management. The phishing email no longer needs to fool you: It just needs to fool your very expensive autocomplete intern.