mlsanityData to 7 Oct 2026

Conversations Risk Detection LLMs in Financial Agents via Multi-Stage Generative Rollout

arXiv 2604.09056 ↗First posted 10 Apr 2026cs.CR cs.CEcited by 0

On the map: LLM agents and tool use → Financial trading and forecasting agents

Abstract (arXiv)

With the rapid adoption of large language models (LLMs) in financial service scenarios, dialogue security detection under high regulatory risk presents significant challenges. Existing methods mainly rely on single-dimensional semantic judgments or fixed rules, making them inadequate for handling multi-turn semantic evolution and complex regulatory clauses; moreover, they lack models specifically designed for financial security detection. To address these issues, this paper proposes FinSec, a four-tier security detection framework for financial agent. FinSec enables structured, interpretable, and end-to-end identification of actual financial risks, incorporating suspicious behavior pattern analysis, delayed risk and adversarial inference, semantic security analysis, and integrated risk-based decision-making. Notably, FinSec significantly enhances the robustness of high-risk dialogue detection while maintaining model utility. Experimental results demonstrate FinSec's leading performance. In terms of overall detection capability, FinSec achieves an F1 score of 90.13%, improving upon baseline models by 6--14 percentage points; its ASR is reduced to 9.09%, markedly lowering the probability of unsafe outputs; and the AUPRC increases to 0.9189 -- an approximate 9.7% gain over general frameworks. Additionally, in balancing utility and safety, FinSec obtains a composite score of 0.9098, delivering robust and efficient protection for financial agent dialogues.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use. Builds on 31 of its 38 references that are in the map. First posted on arXiv 10 Apr 2026.

Builds on

31 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Risk21 Feb
  2. 2024FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models23 May

Other references · 29

  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  3. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  4. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  5. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  6. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  7. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  8. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  9. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
  10. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  11. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  12. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
17 more
  1. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
  2. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  3. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 149
  4. 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 140
  5. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 127
  6. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 124
  7. 2024Improving Multi-Agent Debate with Sparse Communication Topologycited by 103
  8. 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 82
  9. 2025PromptShield: Deployable Detection for Prompt Injection Attackscited by 43
  10. 2025The Truth Becomes Clearer Through Debate! Multi-Agent Systems with Large Language Models Unmask Fake Newscited by 29
  11. 2024Multitask-based Evaluation of Open-Source LLM on Software Vulnerabilitycited by 23
  12. 2025Attack and defense techniques in large language models: A survey and new perspectivescited by 13
  13. 2025The Trust Paradox in LLM-Based Multi-Agent Systems: When Collaboration Becomes a Security Vulnerabilitycited by 11
  14. 2025PII-Bench: Evaluating Query-Aware Privacy Protection Systemscited by 7
  15. 2025SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialoguescited by 7
  16. 2024Can LLMs Learn New Concepts Incrementally without Forgetting?cited by 1
  17. 2025FinVet: A Collaborative Framework of RAG and External Fact-Checking Agents for Financial Misinformation Detectioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FinVault: Benchmarking Financial Agent Safety in Execution-Grounded Environments
  2. 2026Incremental Risk Assessment of Progressive Elder Financial Scams via Instruction-Tuned Small Language Models
  3. 2024FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models
  4. 2025FinHEAR: Human Expertise and Adaptive Risk-Aware Temporal Reasoning for Financial Decision-Making
  5. 2025Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning
  6. 2026SoK: Trading Agents or Market Crashers? Dissecting Robustness and Security Failures in Academic Financial LLM Trading Schemes
  7. 2025Measuring What LLMs Think They Do: SHAP Faithfulness and Deployability on Financial Tabular Classification
  8. 2026FinToolBench: Evaluating LLM Agents for Real-World Financial Tool Use