# Joint Detection of Fraud and Concept Drift inOnline Conversations with LLM-Assisted Judgment

arXiv: https://arxiv.org/abs/2505.07852 · HTML: https://mlsanity.com/paper/2505.07852/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 33 references that are in the map.
- First posted on arXiv 7 May 2025.

## Builds on (most cited first, 7 of 7)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Estimating Continuous Distributions in Bayesian Classifiers](https://mlsanity.com/paper/1302.4964/) (2013)
- [Adaptive Online Incremental Learning for Evolving Data Streams](https://mlsanity.com/paper/2201.01633/) (2022)
- [Are You for Real? Detecting Identity Fraud via Dialogue Interactions](https://mlsanity.com/paper/1908.06820/) (2019)
- [Detecting fake review buyers using network structure: Direct evidence from Amazon](https://mlsanity.com/paper/2410.17507/) (2024)
- [Fraud detection in telephone conversations for financial services using linguistic features](https://mlsanity.com/paper/1912.04748/) (2019)

## Built on by (most cited first, 1 of 1)

- [Domain Knowledge-Enhanced LLMs for Fraud and Concept Drift Detection](https://mlsanity.com/paper/2506.21443/) (2025)

## Abstract (arXiv)

Detecting fake interactions in digital communication platforms remains a challenging and insufficiently addressed problem. These interactions may appear as harmless spam or escalate into sophisticated scam attempts, making it difficult to flag malicious intent early. Traditional detection methods often rely on static anomaly detection techniques that fail to adapt to dynamic conversational shifts. One key limitation is the misinterpretation of benign topic transitions referred to as concept drift as fraudulent behavior, leading to either false alarms or missed threats. We propose a two stage detection framework that first identifies suspicious conversations using a tailored ensemble classification model. To improve the reliability of detection, we incorporate a concept drift analysis step using a One Class Drift Detector (OCDD) to isolate conversational shifts within flagged dialogues. When drift is detected, a large language model (LLM) assesses whether the shift indicates fraudulent manipulation or a legitimate topic change. In cases where no drift is found, the behavior is inferred to be spam like. We validate our framework using a dataset of social engineering chat scenarios and demonstrate its practical advantages in improving both accuracy and interpretability for real time fraud detection. To contextualize the trade offs, we compare our modular approach against a Dual LLM baseline that performs detection and judgment using different language models.