# RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Data

arXiv: https://arxiv.org/abs/2505.19030 · HTML: https://mlsanity.com/paper/2505.19030/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 56 references that are in the map.
- First posted on arXiv 25 May 2025.

## Builds on (most cited first, 30 of 48)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [Reinforcement Learning for Reasoning in Large Language Models with One Training Example](https://mlsanity.com/paper/2504.20571/) (2025)
- [Large Language Models for Mathematical Reasoning: Progresses and Challenges](https://mlsanity.com/paper/2402.00157/) (2024)
- [MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning](https://mlsanity.com/paper/2310.16049/) (2023)
- [Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains](https://mlsanity.com/paper/2503.23829/) (2025)
- [Benchmarking Complex Instruction-Following with Multiple Constraints Composition](https://mlsanity.com/paper/2407.03978/) (2024)
- [InFoBench: Evaluating Instruction Following Ability in Large Language Models](https://mlsanity.com/paper/2401.03601/) (2024)
- [FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models](https://mlsanity.com/paper/2310.20410/) (2023)
- [A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing](https://mlsanity.com/paper/2310.08433/) (2023)
- [CheckEval: A reliable LLM-as-a-Judge framework for evaluating text generation using checklists](https://mlsanity.com/paper/2403.18771/) (2024)
- [Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models](https://mlsanity.com/paper/2406.13542/) (2024)
- [TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generation](https://mlsanity.com/paper/2410.03608/) (2024)
- [A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trends](https://mlsanity.com/paper/2311.10372/) (2023)
- [A Short Survey of Viewing Large Language Models in Legal Aspect](https://mlsanity.com/paper/2303.09136/) (2023)
- [Specific versus General Principles for Constitutional AI](https://mlsanity.com/paper/2310.13798/) (2023)
- [Inverse Constitutional AI: Compressing Preferences into Principles](https://mlsanity.com/paper/2406.06560/) (2024)

## Built on by (most cited first, 12 of 12)

- [Beyond the Trade-off: Self-Supervised Reinforcement Learning for Reasoning Models' Instruction Following](https://mlsanity.com/paper/2508.02150/) (2025)
- [UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering](https://mlsanity.com/paper/2605.30076/) (2026)
- [Precision over Diversity: High-Precision Reward Generalizes to Robust Instruction Following](https://mlsanity.com/paper/2601.04954/) (2026)
- [Instruction Stacking Collapse: A Benchmark and the Capability-Dependent Value of Prompt Compilation](https://mlsanity.com/paper/2608.02639/) (2026)
- [RECAP: Regression Evaluation for Continual Adaptation of Prompts](https://mlsanity.com/paper/2606.06698/) (2026)
- [TinyJudge: Unverifiable Constraint Alignment via Lightweight Specialist Ensembles](https://mlsanity.com/paper/2606.07520/) (2026)
- [UNSPECIFIC: General Constraint Synthesis for Breaking Copy-and-Paste Shortcut in LLM Instruction Following](https://mlsanity.com/paper/2608.09154/) (2026)
- [Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction](https://mlsanity.com/paper/2608.12426/) (2026)
- [Proxy reliance in large language model decisions is uncalibrated to predictive evidence](https://mlsanity.com/paper/2608.22887/) (2026)
- [Mitigating Exploration Bias in RL for Multi-Instruction Following](https://mlsanity.com/paper/2608.23830/) (2026)
- [ReDeck: Step-Level Render-Grounded Refinement for Document-to-Slide Generation](https://mlsanity.com/paper/2609.00194/) (2026)
- [SkillFusion: Towards Reliable Multi-Skill Composition for LLM Agents](https://mlsanity.com/paper/2610.15481/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly expected to tackle complex tasks, driven by their expanding applications and users' growing proficiency in crafting sophisticated prompts. However, as the number of explicitly stated requirements increases (particularly more than 10 constraints), LLMs often struggle to accurately follow such complex instructions, which limits their applicability in complex real-world scenarios. To the best of our knowledge, existing datasets do not exceed 10 constraints per instance. To address this challenge, we propose RECAST, an efficient and scalable framework for synthesizing datasets where each example incorporates far more constraints than those in existing benchmarks, aiming to challenge and extend the boundaries of models' ability to follow complex instructions. These constraints are extracted from real-world prompt-response pairs to ensure practical relevance. Using this framework, we construct RECAST-30K, a large-scale, high-quality dataset comprising 30k instances spanning 19 constraint types. Experimental results demonstrate that models finetuned on RECAST-30K substantially improve in following complex instructions while maintaining their general capabilities without degradation. Moreover, RECAST enables automatic verification of constraint satisfaction via rule-based validators for quantitative constraints and LLM-based validators for qualitative ones; the verifiability provided by RECAST enables the design of reward functions for reinforcement learning, which further boosts model performance on complex and challenging tasks.