# SCRIPTMIND: Crime Script Inference and Cognitive Evaluation for LLM-based Social Engineering Scam Detection System

arXiv: https://arxiv.org/abs/2601.13581 · HTML: https://mlsanity.com/paper/2601.13581/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 41 references that are in the map.
- First posted on arXiv 20 Jan 2026.

## Builds on (most cited first, 8 of 8)

- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [ChatSpamDetector: Leveraging Large Language Models for Effective Phishing Email Detection](https://mlsanity.com/paper/2402.18093/) (2024)
- [PhishAgent: A Robust Multimodal Agent for Phishing Webpage Detection](https://mlsanity.com/paper/2408.10738/) (2024)
- [Personalized Attacks of Social Engineering in Multi-turn Conversations: LLM Agents for Simulation and Detection](https://mlsanity.com/paper/2503.15552/) (2025)
- ["It Warned Me Just at the Right Moment": Exploring LLM-based Real-time Detection of Phone Scams](https://mlsanity.com/paper/2502.03964/) (2025)
- [EXPLICATE: Enhancing Phishing Detection through Explainable AI and LLM-Powered Interpretability](https://mlsanity.com/paper/2503.20796/) (2025)
- [Evaluating LLMs for Police Decision-Making: A Framework Based on Police Action Scenarios](https://mlsanity.com/paper/2601.03553/) (2026)

## Abstract (arXiv)

Social engineering scams increasingly employ personalized, multi-turn deception, exposing the limits of traditional detection methods. While Large Language Models (LLMs) show promise in identifying deception, their cognitive assistance potential remains underexplored. We propose ScriptMind, an integrated framework for LLM-based scam detection that bridges automated reasoning and human cognition. It comprises three components: the Crime Script Inference Task (CSIT) for scam reasoning, the Crime Script-Aware Inference Dataset (CSID) for fine-tuning small LLMs, and the Cognitive Simulation-based Evaluation of Social Engineering Defense (CSED) for assessing real-time cognitive impact. Using 571 Korean phone scam cases, we built 22,712 structured scammer-sequence training instances. Experimental results show that the 11B small LLM fine-tuned with ScriptMind outperformed GPT-4o by 13%, achieving superior performance over commercial models in detection accuracy, false-positive reduction, scammer utterance prediction, and rationale quality. Moreover, in phone scam simulation experiments, it significantly enhanced and sustained users' suspicion levels, improving their cognitive awareness of scams. ScriptMind represents a step toward human-centered, cognitively adaptive LLMs for scam defense.