# UltraIF: Advancing Instruction Following from the Wild

arXiv: https://arxiv.org/abs/2502.04153 · HTML: https://mlsanity.com/paper/2502.04153/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 6 Feb 2025.

## Builds on (most cited first, 14 of 14)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline](https://mlsanity.com/paper/2406.11939/) (2024)
- [Rethinking Benchmark and Contamination for Language Models with Rephrased Samples](https://mlsanity.com/paper/2311.04850/) (2023)
- [Noise Contrastive Alignment of Language Models with Explicit Rewards](https://mlsanity.com/paper/2402.05369/) (2024)
- [Towards a Unified View of Preference Learning for Large Language Models: A Survey](https://mlsanity.com/paper/2409.02795/) (2024)
- [Smaller Language Models Are Better Instruction Evolvers](https://mlsanity.com/paper/2412.11231/) (2024)
- [Evolutionary Contrastive Distillation for Language Model Alignment](https://mlsanity.com/paper/2410.07513/) (2024)

## Built on by (most cited first, 18 of 18)

- [OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment](https://mlsanity.com/paper/2510.07743/) (2025)
- [Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Training](https://mlsanity.com/paper/2602.01511/) (2026)
- [VerIF: Verification Engineering for Reinforcement Learning in Instruction Following](https://mlsanity.com/paper/2506.09942/) (2025)
- [RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Following](https://mlsanity.com/paper/2603.25133/) (2026)
- [Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models](https://mlsanity.com/paper/2506.01413/) (2025)
- [RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Data](https://mlsanity.com/paper/2505.19030/) (2025)
- [Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills](https://mlsanity.com/paper/2607.22529/) (2026)
- [IFDECORATOR: Wrapping Instruction Following Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2508.04632/) (2025)
- [Large Language Models as Computable Approximations to Solomonoff Induction](https://mlsanity.com/paper/2505.15784/) (2025)
- [RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domains](https://mlsanity.com/paper/2605.29156/) (2026)
- [Beyond the Trade-off: Self-Supervised Reinforcement Learning for Reasoning Models' Instruction Following](https://mlsanity.com/paper/2508.02150/) (2025)
- [AHAMask: Reliable Task Specification for Large Audio Language Models without Instructions](https://mlsanity.com/paper/2509.01787/) (2025)
- [ISA-Bench: Benchmarking Instruction Sensitivity for Large Audio Language Models](https://mlsanity.com/paper/2510.23558/) (2025)
- [From Context to EDUs: Faithful and Structured Context Compression via Elementary Discourse Unit Decomposition](https://mlsanity.com/paper/2512.14244/) (2025)
- [SEIF: Self-Evolving Reinforcement Learning for Instruction Following](https://mlsanity.com/paper/2605.07465/) (2026)
- [TCIA: A Task-Centric Instruction Augmentation Method for Instruction Finetuning](https://mlsanity.com/paper/2508.20374/) (2025)
- [LsrIF: Enhancing Logic-Structured Instruction Following of Large Language Models](https://mlsanity.com/paper/2601.06431/) (2026)
- [Cross-Relational Preference Learning for Better LLM Instruction Following](https://mlsanity.com/paper/2608.29352/) (2026)

## Abstract (arXiv)

Instruction-following made modern large language models (LLMs) helpful assistants. However, the key to taming LLMs on complex instructions remains mysterious, for that there are huge gaps between models trained by open-source community and those trained by leading companies. To bridge the gap, we propose a simple and scalable approach UltraIF for building LLMs that can follow complex instructions with open-source data. UltraIF first decomposes real-world user prompts into simpler queries, constraints, and corresponding evaluation questions for the constraints. Then, we train an UltraComposer to compose constraint-associated prompts with evaluation questions. This prompt composer allows us to synthesize complicated instructions as well as filter responses with evaluation questions. In our experiment, for the first time, we successfully align LLaMA-3.1-8B-Base to catch up with its instruct version on 5 instruction-following benchmarks without any benchmark information, using only 8B model as response generator and evaluator. The aligned model also achieved competitive scores on other benchmarks. Moreover, we also show that UltraIF could further improve LLaMA-3.1-8B-Instruct through self-alignment, motivating broader use cases for the method. Our code is available at https://github.com/kkk-an/UltraIF.