mlsanityData to 14 Oct 2026

Learning to Conceal Risk: Controllable Multi-turn Red Teaming for LLMs in the Financial Domain

arXiv 2509.10546 ↗First posted 7 Sep 2025cs.CL cs.AI cs.LGcited by 6

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Large Language Models (LLMs) are increasingly deployed in finance, where unsafe behavior can lead to serious regulatory risks. However, most red-teaming research focuses on overtly harmful content and overlooks attacks that appear legitimate on the surface yet induce regulatory-violating responses. We address this gap by introducing a controllable black-box multi-turn risk-concealed red-teaming framework (CoRT) that progressively conceals surface-level risk while exploiting regulatory-violating behaviors. CoRT contains two key components: (i) a Risk Concealment Attacker (RCA) that generates multi-turn prompts via iterative refinement, and (ii) a Risk Concealment Controller (RCC) that predicts a turn-level Risk Concealment Score (RCS) to steer RCA's follow-up style. We also built a domain-specific benchmark, FinRisk-Bench, with 522 instructions spanning six financial risk categories. Experiments on nine widely used LLMs show that CoRT (RCA) achieves 93.19% average attack success rate (ASR), and CoRT (RCA+RCC) further improves the average ASR to 95.00%. Our code and FinRisk-Bench are available at https://github.com/gcheng128/CoRT.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 28 of its 46 references that are in the map. First posted on arXiv 7 Sep 2025.

Builds on

28 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Understanding and Mitigating Risks of Generative AI in Financial Services25 Apr
  2. 2023AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications14 Nov
  3. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment18 Aug

Other references · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  6. 2023BloombergGPT: A Large Language Model for Financecited by 939
  7. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  8. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  9. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
  10. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
  11. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  12. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 259
13 more
  1. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 222
  2. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 216
  3. 2024TradingAgents: Multi-Agents LLM Financial Trading Frameworkcited by 188
  4. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 170
  5. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 125
  6. 2024A Survey of Large Language Models in Finance (FinLLMs)cited by 79
  7. 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 77
  8. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 70
  9. 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 54
  10. 2024MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialoguecited by 33
  11. 2024FinLlama: Financial Sentiment Classification for Algorithmic Trading Applicationscited by 23
  12. 2025Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Modelscited by 23
  13. 2025Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025TradeTrap: Are LLM-based Trading Agents Truly Reliable and Faithful?cited by 2
  2. 2026Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Servicescited by 1
  3. 2026PolicyAlign: Direct Policy-Based Safety Alignment for Large Language Modelscited by 1
  4. 2026MESA: Prioritizing Vulnerable Communication Channels for Securing Multi-Agent Systemscited by 0
  5. 2026Pragmatic Attack Surface: Vulnerabilities of Implicit Context in Large Language Modelscited by 0
  6. 2026SoK: Trading Agents or Market Crashers? Dissecting Robustness and Security Failures in Academic Financial LLM Trading Schemescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026FinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming
  2. 2026FinSafetyBench: Evaluating LLM Safety in Real-World Financial Scenarios
  3. 2026AMT-X: Phase-Structured Multi-Turn Red-Teaming with Checklist-Gated Evaluation
  4. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teaming
  5. 2025A Red Teaming Roadmap Towards System-Level Safety
  6. 2026FinRT: Distilling Adaptive Red-Teaming Strategies into Reusable Adversarial Generators in Consumer Finance
  7. 2026CHASE: Adversarial Red-Blue Teaming for Improving LLM Safety using Reinforcement Learning
  8. 2026StealthGraph: Exposing Domain-Specific Risks in LLMs through Knowledge-Graph-Guided Harmful Prompt Generation