A Low-Latency Fraud Detection Layer for Detecting Adversarial Interaction Patterns in LLM-Powered Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
Large Language Model (LLM)-powered agents demonstrate strong capabilities in autonomous task execution, tool use, and multi-step reasoning. However, their increasing autonomy also introduces a new attack surface: adversarial interactions can manipulate agent behavior through direct prompt injection, indirect content attacks, and multi-turn escalation strategies. Existing defense strategies focus on prompt-level filtering and rule-based guardrails, which are often insufficient when risk emerges gradually across interaction sequences. In this work, we propose a complementary defense mechanism: a low-latency fraud detection layer for detecting adversarial interaction patterns in LLM-powered agents. Instead of determining whether a single prompt is malicious, our approach models risk over interaction trajectories using structured runtime features derived from prompt characteristics, session dynamics, tool usage, execution context, and fraud-inspired signals. The detection layer can be implemented using lightweight models leading to low-latency real-time deployments. To evaluate the framework, we construct a synthetic corpus of 12,000 multi-turn agent interactions generated from parameterized templates that simulate realistic agentic workflows. Using 42 structured features and an XGBoost classifier, our detector achieves over 9 times faster than LLM-based detectors. Through the experiment and ablation studies, our work suggests that interaction-level behavioral detection should become a core component of deployment-time defense for LLM-powered agents.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 10 of its 42 references that are in the map. First posted on arXiv 1 May 2026.
Builds on
10 of 42 references.
Closest ancestors · 1
Same direction, newest first
Other references · 9
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 177
- 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 149
- 2018Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applicationscited by 127
- 2025Defending Against Prompt Injection with DataFiltercited by 35
- 2026WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoningcited by 12
- 2025Detecting Credit Card Fraud via Heterogeneous Graph Neural Networks with Graph Attentioncited by 8
- 2026A Systematic Literature Review on LLM Defenses Against Prompt Injection and Jailbreaking: Expanding NIST Taxonomycited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Detection of adversarial intent in Human-AI teams using LLMs
- 2024Imprompter: Tricking LLM Agents into Improper Tool Use
- 2026Universal Defenses for Tool-Integrated LLM Agents Against Adversarial Attacks
- 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows
- 2026Defense Against Indirect Prompt Injection via Tool Result Parsing
- 2026Assessing Automated Prompt Injection Attacks in Agentic Environments
- 2025Measuring the Security of Mobile LLM Agents under Adversarial Prompts from Untrusted Third-Party Channels
- 2024Targeting the Core: A Simple and Effective Method to Attack RAG-based Agents via Direct LLM Manipulation