mlsanityData to 14 Oct 2026

Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements

arXiv 2502.12904 ↗First posted 18 Feb 2025cs.CLcited by 32

Not on the map. Why?

Abstract (arXiv)

We introduce Fraud-R1, a benchmark designed to evaluate LLMs' ability to defend against internet fraud and phishing in dynamic, real-world scenarios. Fraud-R1 comprises 8,564 fraud cases sourced from phishing scams, fake job postings, social media, and news, categorized into 5 major fraud types. Unlike previous benchmarks, Fraud-R1 introduces a multi-round evaluation pipeline to assess LLMs' resistance to fraud at different stages, including credibility building, urgency creation, and emotional manipulation. Furthermore, we evaluate 15 LLMs under two settings: 1. Helpful-Assistant, where the LLM provides general decision-making assistance, and 2. Role-play, where the model assumes a specific persona, widely used in real-world agent-based interactions. Our evaluation reveals the significant challenges in defending against fraud and phishing inducement, especially in role-play settings and fake job postings. Additionally, we observe a substantial performance gap between Chinese and English, underscoring the need for improved multilingual fraud detection capabilities.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

26 of 45 references.

References · 26

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  3. 2024A Survey on LLM-as-a-Judgecited by 1,423
  4. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  5. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  6. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
  7. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
  8. 2023An LLM can Fool Itself: A Prompt-Based Adversarial Attackcited by 116
  9. 2024A Survey of Large Language Models in Finance (FinLLMs)cited by 79
  10. 2023Large Language Models for Generative Recommendation: A Survey and Visionary Discussionscited by 73
  11. 2024A Complete Survey on LLM-based AI Chatbotscited by 71
  12. 2024PersonalLLM: Tailoring LLMs to Individual Preferencescited by 60
14 more
  1. 2024eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Datacited by 46
  2. 2024Combining Fine-Tuning and LLM-based Agents for Intuitive Smart Contract Auditing with Justificationscited by 41
  3. 2024Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attackscited by 26
  4. 2024Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Modelscited by 20
  5. 2025Short-length Adversarial Training Helps LLMs Defend Long-length Jailbreak Attacks: Theoretical and Empirical Evidencecited by 14
  6. 2024Dialectical Alignment: Resolving the Tension of 3H and Security Threats of LLMscited by 11
  7. 2024An Explainable Transformer-based Model for Phishing Email Detection: A Large Language Model Approachcited by 9
  8. 2024MONAL: Model Autophagy Analysis for Modeling Human-AI Interactionscited by 7
  9. 2024Novel Interpretable and Robust Web-based AI Platform for Phishing Email Detectioncited by 7
  10. 2024Investigating LLM Applications in E-Commercecited by 7
  11. 2024ResumeFlow: An LLM-facilitated Pipeline for Personalized Resume Generation and Refinementcited by 6
  12. 2024DetoxBench: Benchmarking Large Language Models for Multitask Fraud & Abuse Detectioncited by 5
  13. 2016An intelligent classification model for phishing email detectioncited by 3
  14. 2023Adversarial Data Poisoning for Fake News Detection: How to Make a Model Misclassify a Target News without Modifying Itcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 16.

32 papers.

Sort:

By area:

  1. 2025Understanding the Repeat Curse in Large Language Models from a Feature Perspectivecited by 37
  2. 2025Is Your LLM-Based Multi-Agent a Reliable Real-World Planner? Exploring Fraud Detection in Travel Planningcited by 15
  3. 2025The Automated but Risky Game: Modeling and Benchmarking Agent-to-Agent Negotiations and Transactions in Consumer Marketscited by 15
  4. 2025Understanding How Value Neurons Shape the Generation of Specified Values in LLMscited by 13
  5. 2025PIXEL: Adaptive Steering Via Position-wise Injection with eXact Estimated Levels under Subspace Calibrationcited by 11
  6. 2025JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Modelcited by 10
  7. 2026AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitorcited by 10
  8. 2025When AI Agents Collude Online: Financial Fraud Risks by Collaborative LLM Agents on Social Platformscited by 8
  9. 2026Multi-User Large Language Model Agentscited by 7
  10. 2025COMPKE: Complex Question Answering under Knowledge Editingcited by 6
  11. 2025Can AI Truly Represent Your Voice in Deliberations? A Comprehensive Study of Large-Scale Opinion Aggregation with LLMscited by 6
  12. 2025QualBench: Benchmarking Chinese LLMs with Localized Professional Qualifications for Vertical Domain Evaluationcited by 5
20 more
  1. 2025Tracing and Mitigating Hallucinations in Multimodal LLMs via Dynamic Attention Localizationcited by 4
  2. 2025Understanding Structured Financial Data with LLMs: A Case Study on Fraud Detectioncited by 4
  3. 2026FraudShield: Knowledge Graph Empowered Defense for LLMs against Fraud Attackscited by 4
  4. 2025The Compositional Architecture of Regret in Large Language Modelscited by 3
  5. 2026FinSafetyBench: Evaluating LLM Safety in Real-World Financial Scenarioscited by 3
  6. 2026SAFE-QAQ: End-to-End Slow-Thinking Audio-Text Fraud Detection via Reinforcement Learningcited by 2
  7. 2026The Anatomy of Conversational Scams: A Topic-Based Red Teaming Analysis of Multi-Turn Interactions in LLMscited by 2
  8. 2026The System Prompt Is the Attack Surface: How LLM Agent Configuration Shapes Security and Creates Exploitable Vulnerabilitiescited by 2
  9. 2026PreScam: A Benchmark for Predicting Scam Progression from Early Conversationscited by 2
  10. 2025Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Imagescited by 1
  11. 2025SHERLOCK: Towards Dynamic Knowledge Adaptation in LLM-enhanced E-commerce Risk Managementcited by 1
  12. 2025HarmTransform: Transforming Explicit Harmful Queries into Stealthy via Multi-Agent Debatecited by 1
  13. 2026Harness-MU: A Safe, Governed, and Effective Harness for Multi-User LLM Agentscited by 1
  14. 2026FraudBench: Stress-Testing Policy-Grounded Banking Agents Against Adaptive Fraudcited by 1
  15. 2026A Multi-Turn Framework for Evaluating AI Misuse in Fraud and Cybercrime Scenarioscited by 0
  16. 2026ORACLE: Anticipating Scams from Partial Trajectories in Streaming App Usagecited by 0
  17. 2026Agentic Relationship Harm: Benchmarking and Gating Relational Manipulation in AI Agentscited by 0
  18. 2026AISPA: User-Centric System Prompt Auditing for Large Language Model Applicationscited by 0
  19. 2026Beyond Detection: Evaluating Defensive LLMs Against AI-Generated Social Engineering in Live Turn-by-Turn Interactioncited by 0
  20. 2026TeleAntiFraud 2.0: A Refreshable, Profile-Grounded, and Audio-Based Benchmark for Telecom Fraud Detectioncited by 0