mlsanityData to 7 Oct 2026

MuSC: Improving Complex Instruction Following with Multi-granularity Self-Contrastive Training

arXiv 2502.11541 ↗First posted 17 Feb 2025cs.CL cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

Complex instruction-following with elaborate constraints is imperative for Large Language Models (LLMs). While existing methods have constructed data for complex instruction alignment, they all rely on a more advanced model, especially GPT-4, limiting their application. In this paper, we propose a Multi-granularity Self-Contrastive Training (MuSC) framework, to improve the complex instruction alignment without relying on a stronger model. Our method is conducted on both coarse and fine granularity. On coarse-granularity, we construct constraint-aware preference data based on instruction decomposition and recombination. On fine-granularity, we perform token-aware preference optimization with dynamic token-level supervision. Our method is evaluated on open-sourced models, and experiment results show our method achieves significant improvement on both complex and general instruction-following benchmarks, surpassing previous self-alignment methods.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 45 of its 62 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

45 of 62 references.

References · 45

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2023Let's Verify Step by Stepcited by 3,598
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  11. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  12. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
33 more
  1. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  2. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  3. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
  4. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  5. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  6. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  7. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  8. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  9. 2024Direct Language Model Alignment from Online AI Feedbackcited by 257
  10. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
  11. 2024From r to Q^*: Your Language Model is Secretly a Q-Functioncited by 242
  12. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 210
  13. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 202
  14. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 147
  15. 2024Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedbackcited by 107
  16. 2023Branch-Solve-Merge Improves Large Language Model Evaluation and Generationcited by 104
  17. 2024Reinforcement Learning for LLM Post-Training: A Surveycited by 91
  18. 2023Can Large Language Models Understand Real-World Complex Instructions?cited by 69
  19. 2024CFBench: A Comprehensive Constraints-Following Benchmark for LLMscited by 65
  20. 2024A Comparative Study on Reasoning Patterns of OpenAI's o1 Modelcited by 62
  21. 2024Bootstrapping Language Models with DPO Implicit Rewardscited by 53
  22. 2022ORCA: A Challenging Benchmark for Arabic Language Understandingcited by 46
  23. 2024Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulationcited by 46
  24. 2023Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curationcited by 38
  25. 2024LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraintscited by 34
  26. 2023MUFFIN: Curating Multi-Faceted Instructions for Improving Instruction-Followingcited by 30
  27. 2024I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigmcited by 27
  28. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  29. 2024Beyond Sparse Rewards: Enhancing Reinforcement Learning with Language Model Critique in Text Generationcited by 22
  30. 2024Direct Large Language Model Alignment Through Self-Rewarding Contrastive Prompt Distillationcited by 18
  31. 2023DoG-Instruct: Towards Premium Instruction-Tuning Data via Text-Grounded Instruction Wrappingcited by 16
  32. 2024Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimizationcited by 7
  33. 20242D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervisioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 6.

12 papers.

  1. 2025Incentivizing Reasoning for Advanced Instruction-Following of Large Language Modelscited by 17
  2. 2026VIGIL: Defending LLM Agents Against Tool Stream Injection via Verify-Before-Commitcited by 13
  3. 2025AIR: Complex Instruction Generation via Automatic Iterative Refinementcited by 10
  4. 2025Light-IF: Endowing LLMs with Generalizable Reasoning via Preview and Self-Checking for Complex Instruction Followingcited by 6
  5. 2025IFDECORATOR: Wrapping Instruction Following Reinforcement Learning with Verifiable Rewardscited by 6
  6. 2025Instructions are all you need: Self-supervised Reinforcement Learning for Instruction Followingcited by 6
  7. 2025DecIF: Improving Instruction-Following through Meta-Decompositioncited by 1
  8. 2026Mini-BEHAVIOR-Gran: Revealing U-Shaped Effects of Instruction Granularity on Language-Guided Embodied Agentscited by 1
  9. 2026SEIF: Self-Evolving Reinforcement Learning for Instruction Followingcited by 1
  10. 2026LsrIF: Enhancing Logic-Structured Instruction Following of Large Language Modelscited by 0
  11. 2026STAIF: A Stage-wise Optimization for Complex Instruction Followingcited by 0
  12. 2026Cross-Relational Preference Learning for Better LLM Instruction Followingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization
  2. 2024Enhancing and Assessing Instruction-Following with Fine-Grained Instruction Variants
  3. 2024Align2LLaVA: Cascaded Human and Large Language Model Preference Alignment for Multi-modal Instruction Curation
  4. 2025IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation
  5. 2024Evolutionary Contrastive Distillation for Language Model Alignment
  6. 2025Reverse Preference Optimization for Complex Instruction Following
  7. 2025MAIN: Mutual Alignment Is Necessary for instruction tuning
  8. 2026ScopeIF: Improving Scope-Aware Precise Instruction-Following in Large Language Models via Graded Reward Modeling