mlsanityData to 14 Oct 2026

SEAR: A Multimodal Dataset for Analyzing AR-LLM-Driven Social Engineering Behaviors

arXiv 2505.24458 ↗First posted 30 May 2025cs.AIcited by 2

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

The SEAR Dataset is a novel multimodal resource designed to study the emerging threat of social engineering (SE) attacks orchestrated through augmented reality (AR) and multimodal large language models (LLMs). This dataset captures 180 annotated conversations across 60 participants in simulated adversarial scenarios, including meetings, classes and networking events. It comprises synchronized AR-captured visual/audio cues (e.g., facial expressions, vocal tones), environmental context, and curated social media profiles, alongside subjective metrics such as trust ratings and susceptibility assessments. Key findings reveal SEAR's alarming efficacy in eliciting compliance (e.g., 93.3% phishing link clicks, 85% call acceptance) and hijacking trust (76.7% post-interaction trust surge). The dataset supports research in detecting AR-driven SE attacks, designing defensive frameworks, and understanding multimodal adversarial manipulation. Rigorous ethical safeguards, including anonymization and IRB compliance, ensure responsible use. The SEAR dataset is available at https://github.com/INSLabCN/SEAR-Dataset.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 4 of its 13 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

4 of 13 references.

References · 4

  1. 2024SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Wordscited by 68
  2. 2019Detecting and Characterizing Lateral Phishing at Scalecited by 21
  3. 2016Breaching the Human Firewall: Social engineering in Phishing and Spear-Phishing Emailscited by 7
  4. 2022Privacy-preserving Reflection Rendering for Augmented Realitycited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026UNSEEN: A Cross-Stack LLM Unlearning Defense against AR-LLM Social Engineering Attackscited by 1
  2. 2026PhySE: A Psychological Framework for Real-Time AR-LLM Social Engineering Attackscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025On the Feasibility of Using MultiModal LLMs to Execute AR Social Engineering Attacks
  2. 2024Defending Against Social Engineering Attacks in the Age of LLMs
  3. 2025Personalized Attacks of Social Engineering in Multi-turn Conversations: LLM Agents for Simulation and Detection
  4. 2026Evaluating Social Engineering Risks in AI-based Interaction using Biometrics and a Gaming Setup
  5. 2026The Anatomy of Conversational Scams: A Topic-Based Red Teaming Analysis of Multi-Turn Interactions in LLMs
  6. 2025Can MLLMs Read the Room? A Multimodal Benchmark for Assessing Deception in Multi-Party Social Interactions
  7. 2024The Shadow of Fraud: The Emerging Danger of AI-powered Social Engineering and its Possible Cure
  8. 2025Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models