mlsanityData to 7 Oct 2026

IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization

arXiv 2411.06208 ↗First posted 9 Nov 2024cs.CL cs.AIcited by 30

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

In the realm of large language models (LLMs), the ability of models to accurately follow instructions is paramount as more agents and applications leverage LLMs for construction, where the complexity of instructions are rapidly increasing. However, on the one hand, there is only a certain amount of complex instruction evaluation data; on the other hand, there are no dedicated algorithms to improve the ability to follow complex instructions. To this end, this paper introduces TRACE, a benchmark for improving and evaluating the complex instructionfollowing ability, which consists of 120K training data and 1K evaluation data. Furthermore, we propose IOPO (Input-Output Preference Optimization) alignment method which takes both input and output preference pairs into consideration, where LLMs not only rapidly align with response preferences but also meticulously explore the instruction preferences. Extensive experiments on both in-domain and outof-domain datasets confirm the effectiveness of IOPO, showing 8.15%, 2.18% improvements on in-domain data and 6.29%, 3.13% on outof-domain data compared to SFT and DPO respectively.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 24 of its 40 references that are in the map. First posted on arXiv 9 Nov 2024.

Builds on

24 of 40 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Reward23 May
  2. 2023Preference Ranking Optimization for Human Alignment30 Jun
  3. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 21

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  4. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  5. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  6. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  7. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  8. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  9. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  10. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  11. 2023OpenAGI: When LLM Meets Domain Expertscited by 196
  12. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 147
9 more
  1. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QAcited by 121
  2. 2023Wider and Deeper LLM Networks are Fairer LLM Evaluatorscited by 91
  3. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  4. 2023Can Large Language Models Understand Real-World Complex Instructions?cited by 69
  5. 2024CFBench: A Comprehensive Constraints-Following Benchmark for LLMscited by 65
  6. 2024A Taxonomy for Human-LLM Interaction Modes: An Initial Explorationcited by 38
  7. 2021Learning to Generate Task-Specific Adapters from Task Descriptioncited by 28
  8. 2024The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trendscited by 11
  9. 2024Language Models can Evaluate Themselves via Probability Discrepancycited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 20; 2026: 10.

30 papers.

Sort:

By area:

  1. 2025VerIF: Verification Engineering for Reinforcement Learning in Instruction Followingcited by 43
  2. 2025MM-IFEngine: Towards Multimodal Instruction Followingcited by 41
  3. 2025A Survey of Direct Preference Optimizationcited by 35
  4. 2025Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generationcited by 21
  5. 2026RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Followingcited by 19
  6. 2025Incentivizing Reasoning for Advanced Instruction-Following of Large Language Modelscited by 17
  7. 2025SOTOPIA-Omega: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agentscited by 16
  8. 2025MulDimIF: A Multi-Dimensional Constraint Framework for Evaluating and Improving Instruction Following in Large Language Modelscited by 13
  9. 2025SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimizationcited by 10
  10. 2025EvaLearn: Quantifying the Learning Capability and Efficiency of LLMs via Sequential Problem Solvingcited by 10
  11. 2025Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMscited by 9
  12. 2025EIFBENCH: Extremely Complex Instruction Following Benchmark for Large Language Modelscited by 9
18 more
  1. 2025Routine: A Structural Planning Framework for LLM Agent System in Enterprisecited by 9
  2. 2026IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluationcited by 8
  3. 2025Replay Failures as Successes: Sample-Efficient Reinforcement Learning for Instruction Followingcited by 7
  4. 2025Light-IF: Endowing LLMs with Generalizable Reasoning via Preview and Self-Checking for Complex Instruction Followingcited by 6
  5. 2025IFDECORATOR: Wrapping Instruction Following Reinforcement Learning with Verifiable Rewardscited by 6
  6. 2025IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluationcited by 4
  7. 2025Reverse Preference Optimization for Complex Instruction Followingcited by 2
  8. 2025InstructEngine: Instruction-driven Text-to-Image Alignmentcited by 1
  9. 2025DecIF: Improving Instruction-Following through Meta-Decompositioncited by 1
  10. 2026CCR-Bench: A Comprehensive Benchmark for Evaluating LLMs on Complex Constraints, Control Flows, and Real-World Casescited by 1
  11. 2026Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Designcited by 1
  12. 2026One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulnesscited by 1
  13. 2026SEIF: Self-Evolving Reinforcement Learning for Instruction Followingcited by 1
  14. 2025Knots: A Large-Scale Multi-Agent Enhanced Expert-Annotated Dataset and LLM Prompt Optimization for NOTAM Semantic Parsingcited by 0
  15. 2026STAIF: A Stage-wise Optimization for Complex Instruction Followingcited by 0
  16. 2026UNSPECIFIC: General Constraint Synthesis for Breaking Copy-and-Paste Shortcut in LLM Instruction Followingcited by 0
  17. 2026Cross-Relational Preference Learning for Better LLM Instruction Followingcited by 0
  18. 2026ScopeIF: Improving Scope-Aware Precise Instruction-Following in Large Language Models via Graded Reward Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing and Assessing Instruction-Following with Fine-Grained Instruction Variants
  2. 2024Align2LLaVA: Cascaded Human and Large Language Model Preference Alignment for Multi-modal Instruction Curation
  3. 2024FIPO: Free-form Instruction-oriented Prompt Optimization with Preference Dataset and Modular Fine-tuning Schema
  4. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
  5. 2025Got Compute, but No Data: Lessons From Post-training a Finnish LLM
  6. 2025Reward-aware Preference Optimization: A Unified Mathematical Framework for Model Alignment
  7. 2024Triple Preference Optimization: Achieving Better Alignment using a Single Step Optimization
  8. 2025MuSC: Improving Complex Instruction Following with Multi-granularity Self-Contrastive Training