mlsanityData to 14 Oct 2026

REFLECTOR: Internalizing Step-wise Reflection against Indirect Jailbreak

arXiv 2605.20654 ↗First posted 20 May 2026cs.LG cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

While Large Language Models (LLMs) demonstrate remarkable capabilities, they remain susceptible to sophisticated, multi-step jailbreak attacks that circumvent conventional surface-level safety alignment by exploiting the internal generation process. To address these vulnerabilities, we propose Reflector, a principled two-stage framework that internalizes self-reflection within the generation trajectory. Reflector first leverages teacher-guided generation to produce high-quality reflection data for supervised fine-tuning (SFT), establishing structured reflection patterns. It subsequently uses Reinforcement Learning (RL) with outcome-driven and reward-validity supervision to instill robust, autonomous self-reflection capabilities. Empirical results show that Reflector achieves Defense Success Rates (DSR) exceeding 90% against complex indirect attacks while generalizing robustly across diverse threat scenarios. Notably, the framework enhances both task-specific and general utility, yielding a 5.85% gain on GSM8K alongside improved performance on knowledge-intensive benchmarks. By internalizing trajectory-level safety, Reflector overcomes the fundamental limitations of surface alignment without significant computational overhead, offering an efficient and scalable solution for the development of safe and capable LLMs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 55 of its 61 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

55 of 61 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Lifelong Safety Alignment for Language Models26 May
  2. 2025STAIR: Improving Safety Alignment with Introspective Reasoning4 Feb
  3. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep10 Jun
  4. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications7 Feb
  5. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedback19 Oct
  6. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  8. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 47

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  9. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  12. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
35 more
  1. 2016Generative Adversarial Imitation Learningcited by 1,597
  2. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  3. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  4. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  5. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  6. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  7. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  8. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  9. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  10. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
  11. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  12. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  13. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  14. 2024A StrongREJECT for Empty Jailbreakscited by 385
  15. 2024Measuring short-form factuality in large language modelscited by 373
  16. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  17. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 264
  18. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 259
  19. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 232
  20. 2023Using an LLM to Help With Code Understandingcited by 228
  21. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  22. 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationcited by 193
  23. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
  24. 2025Qwen3Guard Technical Reportcited by 155
  25. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 130
  26. 2024Simulating Classroom Education with LLM-Empowered Agentscited by 127
  27. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstructioncited by 107
  28. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 86
  29. 2024Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Modelscited by 72
  30. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 70
  31. 2023Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injectioncited by 59
  32. 2025Improve LLM-as-a-Judge Ability as a General Abilitycited by 37
  33. 2025On the Fundamental Limits of LLMs at Scalecited by 18
  34. 2025Think Twice, Generate Once: Safeguarding by Progressive Self-Reflectioncited by 11
  35. 2026Understanding and Preserving Safety in Fine-Tuned LLMscited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Self-Reflection Fine-Tuning: Enhancing Agent Security against Prompt Injection Attacks from Failure Experience
  2. 2026Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection
  3. 2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender
  4. 2026Be Your Own Red Teamer: Safety Alignment via Self-Play and Reflective Experience Replay
  5. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
  6. 2026Compositional Jailbreaking: An Empirical Analysis of Mutator Chain Interactions in Aligned LLMs
  7. 2025Reflection-Driven Control for Trustworthy Code Agents
  8. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization