ScopeIF: Improving Scope-Aware Precise Instruction-Following in Large Language Models via Graded Reward Modeling
On the map: LLM alignment / RLHF / preference optimisation → Process reward models for reasoning
Abstract (arXiv)
Precise instruction-following is a fundamental ability of large language models (LLMs), requiring their outputs to strictly satisfy objective constraints in input instructions. In complex application scenarios, these constraints often possess diverse scopes that govern specific response segments rather than the entire output. However, existing optimization methods often neglect constraint scope during data construction and rely on binary per-constraint rewards, yielding limited data diversity and sparse supervision for complex constraints. To this end, we propose ScopeIF, a novel training framework for scope-aware precise instruction-following. We first introduce a unified schema that factorizes objective constraints into three decoupled dimensions: Scope, Target, and Range. Grounded in this schema, we construct ScopeInstruct, a large-scale instruction dataset with diverse scope-aware constraints, and combine tool-grounded verification with graded reward modeling to quantify the violation degree of each constraint, providing dense supervision for policy optimization. Extensive experiments demonstrate that ScopeIF consistently outperforms existing methods, particularly on complex scope-aware constraints, while preserving general capabilities. Notably, it enables optimized Qwen3-4B and 8B models to rival or surpass strong frontier models such as Gemini-2.5-Pro and DeepSeek-V3.2, establishing an effective paradigm for advancing scope-aware instruction-following. Our code and data are available at https://github.com/thu-coai/ScopeIF.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 42 of its 53 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
42 of 53 references.
References · 42
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2023Let's Verify Step by Stepcited by 3,598
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2023A Survey of Large Language Modelscited by 2,537
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
30 more
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
- 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
- 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
- 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
- 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
- 2025Generalizing Verifiable Instruction Followingcited by 178
- 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 147
- 2018Generating Informative and Diverse Conversational Responses via Adversarial Information Maximizationcited by 114
- 2024FOFO: A Benchmark to Evaluate LLMs' Format-Following Capabilitycited by 79
- 2025Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systemscited by 49
- 2025VerIF: Verification Engineering for Reinforcement Learning in Instruction Followingcited by 43
- 2023Large Language Model Instruction Following: A Survey of Progresses and Challengescited by 31
- 2024IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimizationcited by 30
- 2024From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Modelscited by 27
- 2024Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Modelscited by 26
- 2024ChatGLM-RLHF: Practices of Aligning Large Language Models with Human Feedbackcited by 25
- 2024SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Modelscited by 23
- 2024Prompt-Based Length Controlled Generation with Multiple Control Typescited by 19
- 2025Incentivizing Reasoning for Advanced Instruction-Following of Large Language Modelscited by 17
- 2025Verifiable Format Control for Large Language Model Generationscited by 16
- 2025MulDimIF: A Multi-Dimensional Constraint Framework for Evaluating and Improving Instruction Following in Large Language Modelscited by 13
- 2024Constraint Back-translation Improves Complex Instruction Following of Large Language Modelscited by 11
- 2025LIFEBench: Evaluating Length Instruction Following in Large Language Modelscited by 10
- 2024ReFF: Reinforcing Format Faithfulness in Language Models across Varied Taskscited by 9
- 2026IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluationcited by 8
- 2025Replay Failures as Successes: Sample-Efficient Reinforcement Learning for Instruction Followingcited by 7
- 2025IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluationcited by 4
- 2026ImpRIF: Stronger Implicit Reasoning Leads to Better Complex Instruction Followingcited by 3
- 2026LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Modelscited by 1
- 2026IFHierBench: Hierarchical Instruction Following for Large Language Modelscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024ROSE: A Reward-Oriented Data Selection Framework for LLM Task-Specific Instruction Tuning
- 2024Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models
- 2026TinyJudge: Unverifiable Constraint Alignment via Lightweight Specialist Ensembles
- 2026STAIF: A Stage-wise Optimization for Complex Instruction Following
- 2025BLEUBERI: BLEU is a surprisingly effective reward for instruction following
- 2024Benchmarking Large Language Models on Controllable Generation under Diversified Instructions
- 2026SCOPE: Self-Play via Co-Evolving Policies for Open-Ended Tasks
- 2023Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks