2 min read
|
Saved February 14, 2026
|
Copied!
Do you care about this?
The article discusses a study on how AI models can be manipulated to create phishing emails that target elderly individuals. Conducted in partnership with Reuters, the research found that 11% of participants fell for at least one phishing attempt, highlighting the growing threat of AI in scams. The authors also address the broader implications of AI misuse in fraud.
If you do, here's more
Fred Heiding and his team partnered with Reuters to investigate how AI can be exploited in scams targeting the elderly. Their research aimed to understand the effectiveness of AI-generated phishing attempts, revealing alarming statistics about vulnerabilities within this demographic. In their study, they reached out to two senior organizations in California, enrolling participants who willingly received phishing emails crafted using jailbreak techniques on various AI models. Out of 108 participants, 11% fell victim to at least one phishing email, with the most effective message prompting a 9% click-through rate on a malicious link.
The study highlighted that while simple jailbreaks were effective against AI systems from Meta and Gemini, ChatGPT and Claude showed more resilience. The broader investigation led by Reuters uncovered that organized crime rings in Southeast Asia use AI tools to enhance their scams, with victims coerced into working in scam factories. These insights were further emphasized by a Senate hearing prompted by their findings, particularly concerning the risks AI chatbots pose to older Americans.
Heiding's research fills a significant gap in understanding the potential harms of AI misuse, moving from theoretical jailbreaking to real-world impacts. They have also begun exploring voice scams, indicating an ongoing commitment to address this pressing issue. Their findings were published in a paper on arXiv and accepted at the AAAI conference, underscoring the need for more human studies on AI's effects. The work received support from Manifund and was recommended by Neel Nanda.
Questions about this article
No questions yet.