# Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes

arXiv: https://arxiv.org/abs/2606.27210 · HTML: https://mlsanity.com/paper/2606.27210/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 25 Jun 2026.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)
- Knowledge distillation / [On-policy and self-distillation for LLMs](https://mlsanity.com/distill/on-policy-and-self-distillation-for-llms/)

## Builds on (most cited first, 21 of 21)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs](https://mlsanity.com/paper/2406.18495/) (2024)
- [NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails](https://mlsanity.com/paper/2310.10501/) (2023)
- [A Holistic Approach to Undesired Content Detection in the Real World](https://mlsanity.com/paper/2208.03274/) (2022)
- [Large Language Model Alignment: A Survey](https://mlsanity.com/paper/2309.15025/) (2023)
- [ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation](https://mlsanity.com/paper/2310.17389/) (2023)
- [Intention Analysis Makes LLMs A Good Jailbreak Defender](https://mlsanity.com/paper/2401.06561/) (2024)
- [Spectral Editing of Activations for Large Language Model Alignment](https://mlsanity.com/paper/2405.09719/) (2024)
- [SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering](https://mlsanity.com/paper/2408.11491/) (2024)
- [SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signals](https://mlsanity.com/paper/2502.01042/) (2025)
- [Safety Through Reasoning: An Empirical Study of Reasoning Guardrail Models](https://mlsanity.com/paper/2505.20087/) (2025)
- [Lightweight Safety Guardrails Using Fine-tuned BERT Embeddings](https://mlsanity.com/paper/2411.14398/) (2024)
- [ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning](https://mlsanity.com/paper/2507.11500/) (2025)
- [LLM Performance Predictors: Learning When to Escalate in Hybrid Human-AI Moderation Systems](https://mlsanity.com/paper/2601.07006/) (2026)

## Built on by (most cited first, 1 of 1)

- [Safeguards Based on Copyable Context Cannot Provide Reliable Safety for LLMs](https://mlsanity.com/paper/2607.27951/) (2026)

## Abstract (arXiv)

We argue that safety classifiers should model user intent as an explicit signal between the prompt and the final label. To study this, we introduce AIMS, a human-annotated dataset of 1,724 difficult safety prompts, each paired with an intent description and harm label. We use AIMS to evaluate intent-aware training across supervised fine-tuning, preference learning, reasoning distillation, and reinforcement learning. Despite its size, AIMS enables competitive safety classifiers across training regimes: DPO from model-generated intent errors improves over SFT, and intent-conditioned distillation outperforms reasoning-only distillation in most teacher-student pairs. Most notably, directly rewarding intent faithfulness with GRPO yields the strongest average performance across five external safety benchmarks, while our intent-aware models form the inference latency-F1 Pareto frontier. These results show that faithful intent modeling is a compact, high-quality supervision signal for more robust safety classifiers.