# PsyScam: A Benchmark for Psychological Techniques in Real-World Scams

arXiv: https://arxiv.org/abs/2505.15017 · HTML: https://mlsanity.com/paper/2505.15017/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2025.

## Built on by (most cited first, 8 of 8)

- [PreScam: A Benchmark for Predicting Scam Progression from Early Conversations](https://mlsanity.com/paper/2605.12243/) (2026)
- [MMHBench: A Multi-Perspective Benchmark for Mental Health Understanding in Long-Form Videos](https://mlsanity.com/paper/2607.27895/) (2026)
- [OPBench: A Graph Benchmark to Combat the Opioid Crisis](https://mlsanity.com/paper/2602.14602/) (2026)
- [Targeted Bit-Flip Attacks on LLM-Based Agents](https://mlsanity.com/paper/2603.10042/) (2026)
- [Vishing-Tactics-Bench: Forecasting Exploitation Trajectories in Voice Phishing Calls](https://mlsanity.com/paper/2609.07151/) (2026)
- [Guarding Against Malicious Biased Threats (GAMBiT) Experiments: Revealing Cognitive Bias in Human-Subjects Red-Team Cyber Range Operations](https://mlsanity.com/paper/2508.20963/) (2025)
- [BEACON: A Unified Behavioral-Tactical Framework for Explainable Cybercrime Analysis with Large Language Models](https://mlsanity.com/paper/2512.06555/) (2025)
- [AI YOU Town: Make Friends and Money with Your Digital Twin](https://mlsanity.com/paper/2607.10539/) (2026)

## Abstract (arXiv)

Over the years, online scams have grown dramatically, with nearly 50% of global consumers encountering scam attempts each week. These scams cause not only significant financial losses to individuals and businesses, but also lasting psychological trauma, largely due to scammers' strategic employment of psychological techniques (PTs) to manipulate victims. Meanwhile, scammers continually evolve their tactics by leveraging advances in Large Language Models (LLMs) to generate diverse scam variants that easily bypass existing defenses. To address this pressing problem, we introduce PsyScam, a benchmark designed to systematically capture the PTs employed in real-world scam reports, and investigate how LLMs can be utilized to generate variants of scams based on the PTs and the contexts provided by these scams. Specifically, we collect a wide range of scam reports and ground its annotations of employed PTs in well-established cognitive and psychological theories. We further demonstrate LLMs' capabilities in generating through two downstream tasks: scam completion, and scam augmentation. Experimental results show that PsyScam presents significant challenges to existing models in both detecting and generating scam content based on the PTs used by real-world scammers. Our code and dataset are available.