# Synthetic Trust Attacks: Modeling How Generative AI Manipulates Human Decisions in Social Engineering Fraud

arXiv: https://arxiv.org/abs/2604.04951 · HTML: https://mlsanity.com/paper/2604.04951/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 13 references that are in the map.
- First posted on arXiv 2 Apr 2026.

## Builds on (most cited first, 2 of 2)

- [DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection](https://mlsanity.com/paper/2001.00179/) (2020)
- [Audio Deepfake Detection: A Survey](https://mlsanity.com/paper/2308.14970/) (2023)

## Abstract (arXiv)

Imagine receiving a video call from your CFO, surrounded by colleagues, asking you to urgently authorise a confidential transfer. You comply. Every person on that call was fake, and you just lost 25 million. This is not a hypothetical. It happened in Hong Kong in January 2024, and it is becoming the template for a new generation of fraud. AI has not invented a new crime. It has industrialised an ancient one: the manufacture of trust. This paper proposes Synthetic Trust Attacks (STAs) as a formal threat category and introduces STAM, the Synthetic Trust Attack Model, an eight-stage operational framework covering the full attack chain from adversary reconnaissance through post-compliance leverage. The core argument is this: existing defenses target synthetic media detection, but the real attack surface is the victim's decision. When human deepfake detection accuracy sits at approximately 55.5%, barely above chance, and LLM scam agents achieve 46% compliance versus 18% for human operators while evading safety filters entirely, the perception layer has already failed. Defense must move to the decision layer. We present a five-category Trust-Cue Taxonomy, a reproducible 17-field Incident Coding Schema with a pilot-coded example, and four falsifiable hypotheses linking attack structure to compliance outcomes. The paper further operationalizes the author's practitioner-developed Calm, Check, Confirm protocol as a research-grade decision-layer defense. Synthetic credibility, not synthetic media, is the true attack surface of the AI fraud era.