mlsanityData to 14 Oct 2026

Don't Walk the Line: Boundary Guidance for Filtered Generation

arXiv 2510.11834 ↗First posted 13 Oct 2025cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Generative models are increasingly paired with safety classifiers that filter harmful or undesirable outputs. A common strategy is to fine-tune the generator to reduce the probability of being filtered, but this can be suboptimal: it often pushes the model toward producing samples near the classifier's decision boundary, increasing both false positives and false negatives. We propose Boundary Guidance, a reinforcement learning fine-tuning method that explicitly steers generation away from the classifier's margin. On a benchmark of jailbreak, ambiguous, and longcontext prompts, Boundary Guidance improves both the safety and the utility of outputs, as judged by LLM-as-a-Judge evaluations. Comprehensive ablations across model scales and reward designs demonstrate the robustness of our approach.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 54 references that are in the map. First posted on arXiv 13 Oct 2025.

Builds on

38 of 54 references.

References · 38

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  10. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  11. 2020Learning to summarize from human feedbackcited by 2,588
  12. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
26 more
  1. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  2. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  3. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  4. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  5. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  6. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  7. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
  8. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  9. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
  10. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  11. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
  12. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  13. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 244
  14. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
  15. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 223
  16. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 214
  17. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 181
  18. 2025RewardBench 2: Advancing Reward Model Evaluationcited by 150
  19. 2022Adversarial Training for High-Stakes Reliabilitycited by 60
  20. 2025From Hard Refusals to Safe-Completions: Toward Output-Centric Safety Trainingcited by 52
  21. 2024Enhancing LLM Safety via Constrained Direct Preference Optimizationcited by 49
  22. 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Modelscited by 37
  23. 2025SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safetycited by 37
  24. 2024Gradient-Based Language Model Red Teamingcited by 34
  25. 2024Stepwise Alignment for Constrained Language Model Policy Optimizationcited by 30
  26. 2024Automated Adversarial Discovery for Safety Classifierscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking
  2. 2023Guided Flows for Generative Modeling and Decision Making
  3. 2026Provably Safe Generative Sampling with Constricting Barrier Functions
  4. 2026On the Failure of Boundary-Seeking Distillation in Bottlenecked Generative Architectures
  5. 2023Bridging the Gap: Addressing Discrepancies in Diffusion Model Training for Classifier-Free Guidance
  6. 2025Diffusion Guidance Is a Controllable Policy Improvement Operator
  7. 2026Guidance Is Not a Hyperparameter: Learning Dynamic Control in Diffusion Language Models
  8. 2026Safe Image Generation via Reinforcement Learning