# Before the Warning Comes Too Late: Incremental Phone-Scam Detection from Speech

arXiv: https://arxiv.org/abs/2609.20223 · HTML: https://mlsanity.com/paper/2609.20223/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 32 references that are in the map.
- First posted on arXiv 28 Jul 2026.

## Builds on (most cited first, 10 of 10)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [Overview and Evaluation of Sound Event Localization and Detection in DCASE 2019](https://mlsanity.com/paper/2009.02792/) (2020)
- ["It Warned Me Just at the Right Moment": Exploring LLM-based Real-time Detection of Phone Scams](https://mlsanity.com/paper/2502.03964/) (2025)
- [TeleAntiFraud-28k: An Audio-Text Slow-Thinking Dataset for Telecom Fraud Detection](https://mlsanity.com/paper/2503.24115/) (2025)
- [GAT-COBO: Cost-Sensitive Graph Neural Network for Telecom Fraud Detection](https://mlsanity.com/paper/2303.17334/) (2023)
- [Improving weakly supervised sound event detection with self-supervised auxiliary tasks](https://mlsanity.com/paper/2106.06858/) (2021)
- [Detecting Mental Manipulation in Speech via Synthetic Multi-Speaker Dialogue](https://mlsanity.com/paper/2601.08342/) (2026)

## Abstract (arXiv)

We study weakly supervised incremental telecom fraud detection from raw telephone audio, where training provides only conversation-level labels and predictions must be updated before a call ends. We introduce StreamFraudNet, which processes incoming audio through overlapping bounded-context windows using a frozen self-supervised speech encoder, recurrent temporal modeling, and learned aggregation of latent window scores. On a controlled English benchmark, StreamFraudNet achieves a ROC--AUC of (0.9953), significantly outperforming acoustic and mean-pooling baselines while remaining competitive with strong global temporal models. The model produces its first prediction after 10 seconds of audio, updates every 2 seconds, and operates faster than real time on the evaluated server hardware. Ablations identify recurrent temporal context as the principal contributor to performance. These results demonstrate that fraud risk can be scored incrementally from raw speech without transcripts or temporal annotations, while highlighting the need for latency-aware training to improve early prediction.