# Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models

arXiv: https://arxiv.org/abs/2501.04945 · HTML: https://mlsanity.com/paper/2501.04945/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jan 2025.

## Built on by (most cited first, 12 of 12)

- [VerIF: Verification Engineering for Reinforcement Learning in Instruction Following](https://mlsanity.com/paper/2506.09942/) (2025)
- [RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Following](https://mlsanity.com/paper/2603.25133/) (2026)
- [Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models](https://mlsanity.com/paper/2506.01413/) (2025)
- [Light-IF: Endowing LLMs with Generalizable Reasoning via Preview and Self-Checking for Complex Instruction Following](https://mlsanity.com/paper/2508.03178/) (2025)
- [Instructions are all you need: Self-supervised Reinforcement Learning for Instruction Following](https://mlsanity.com/paper/2510.14420/) (2025)
- [Beyond the Trade-off: Self-Supervised Reinforcement Learning for Reasoning Models' Instruction Following](https://mlsanity.com/paper/2508.02150/) (2025)
- [UltraGen: Extremely Fine-grained Controllable Generation via Attribute Reconstruction and Global Preference Optimization](https://mlsanity.com/paper/2502.12375/) (2025)
- [Format Inertia: A Failure Mechanism of LLMs in Medical Pre-Consultation](https://mlsanity.com/paper/2510.01688/) (2025)
- [Financial Instruction Following Evaluation (FIFE)](https://mlsanity.com/paper/2512.08965/) (2025)
- [Disentangling Curriculum Learning in NLP: Towards a Unifying Taxonomy](https://mlsanity.com/paper/2607.18984/) (2026)
- [TA-RAG: Tone Awareness as a Design Imperative for Retrieval-Augmented Generation](https://mlsanity.com/paper/2608.06672/) (2026)
- [Cross-Relational Preference Learning for Better LLM Instruction Following](https://mlsanity.com/paper/2608.29352/) (2026)

## Abstract (arXiv)

It is crucial for large language models (LLMs) to follow instructions that involve multiple constraints. However, it is an unexplored area to enhance LLMs' ability to follow soft constraints. To bridge the gap, we initially design a pipeline to construct datasets with high-quality outputs automatically. Additionally, to fully utilize the positive and negative samples generated during the data construction process, we choose Direct Preference Optimization (DPO) as the training method. Furthermore, taking into account the difficulty of soft constraints indicated by the number of constraints, we design a curriculum learning training paradigm based on the constraint quantity. We experimentally evaluate the effectiveness of our methods in improving LLMs' soft constraint following ability and analyze the factors driving the improvements.The datasets and code are publicly available at https://github.com/Rainier-rq/FollowSoftConstraint.