mlsanityData to 7 Oct 2026

TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety

arXiv 2610.01323 ↗First posted 1 Oct 2026cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Safety-aligned large language models (LLMs) often refuse a harmful request but comply once the same goal is spread over several turns. Preference objectives score whole responses to single prompts, so their training loss alone cannot control risk on unseen histories. Our analysis gives sufficient conditions under which suppression at supervised single-turn contexts yields a bound on multi-turn trajectory risk. The bound accounts for coverage, transfer slack, and leakage, and characterizes contraction relative to a base-policy risk budget evaluated on the trained policy's contexts. TRACE (Trajectory Return Attribution and Contrastive Erasure) turns this principle into a token-level objective. On the safe response, each token is weighted by the discounted return of a refusal-attributable advantage. The advantage compares a frozen reference model with its refusal-ablated copy, allowing earlier response tokens to receive credit from later refusal-related evidence. At high-gap positions on rejected responses, TRACE combines the observed token with policy-selected alternatives in the erasure target. A gradient-norm penalty replaces the retain set. Across five open-weight models and seven multi-turn attacks, TRACE gives the lowest attack success rate (ASR) in all 35 model and attack pairs, while the model utility evaluated on MMLU and HellaSwag drop by at most 1.23 points. Source code can be found in the supplemental material.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 48 of its 59 references that are in the map. First posted on arXiv 1 Oct 2026.

Builds on

48 of 59 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026Improving Safety Alignment via Balanced Direct Preference Optimization24 Mar
  2. 2026State-Dependent Safety Failures in Multi-Turn Language Model Interaction15 Mar
  3. 2025Mitigating the Safety Alignment Tax with Null-Space Constrained Policy Optimization12 Dec
  4. 2025SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safety26 May
  5. 2025Improving LLM Safety Alignment with Dual-Objective Optimization5 Mar
  6. 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable1 Mar
  7. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal20 Jun
  8. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep10 Jun
  9. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  10. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 37

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  9. 2025Gemma 3 Technical Reportcited by 1,686
  10. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  11. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  12. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
25 more
  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  2. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
  3. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  4. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  5. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  6. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 484
  7. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  8. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  9. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  10. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 256
  11. 2024From r to Q^*: Your Language Model is Secretly a Q-Functioncited by 242
  12. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearningcited by 160
  13. 2026Qwen3.5-Omni Technical Reportcited by 146
  14. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  15. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
  16. 2022Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learningcited by 122
  17. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
  18. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Modelscited by 61
  19. 2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 51
  20. 2024TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weightscited by 42
  21. 2025Diverse Preference Learning for Capabilities and Alignmentcited by 25
  22. 2025Multi-Turn Jailbreaks Are Simpler Than They Seemcited by 15
  23. 2025Which Retain Set Matters for LLM Unlearning? A Case Study on Entity Unlearningcited by 14
  24. 2025Uni-DPO: A Unified Paradigm for Dynamic Preference Optimization of LLMscited by 9
  25. 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safetycited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Refusal Localizes, the Damage Relocates: Safety Layers Under Few-Sample Fine-Tuning
  2. 2026Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures
  3. 2026When Choices Become Risks: Safety Failures of Large Language Models under Multiple-Choice Constraints
  4. 2026The Geometry of Harmfulness in Multi-Turn Attacks
  5. 2026Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment
  6. 2026When Safety Routing Breaks: Understanding Alignment Fragility under Benign Fine-Tuning
  7. 2026Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
  8. 2026Localizing Safety Alignment: MLP Layers and Mid-Network Blocks Encode Refusal Behavior in Large Language Models