# Suri: Multi-constraint Instruction Following for Long-form Text Generation

arXiv: https://arxiv.org/abs/2406.19371 · HTML: https://mlsanity.com/paper/2406.19371/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 39 of its 62 references that are in the map.
- First posted on arXiv 27 Jun 2024.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 30 of 39)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [ORPO: Monolithic Preference Optimization without Reference Model](https://mlsanity.com/paper/2403.07691/) (2024)
- [A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories](https://mlsanity.com/paper/1604.01696/) (2016)
- [How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources](https://mlsanity.com/paper/2306.04751/) (2023)
- [Effective Long-Context Scaling of Foundation Models](https://mlsanity.com/paper/2309.16039/) (2023)
- [Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization](https://mlsanity.com/paper/2210.01241/) (2022)
- [SCROLLS: Standardized CompaRison Over Long Language Sequences](https://mlsanity.com/paper/2201.03533/) (2022)
- [With Little Power Comes Great Responsibility](https://mlsanity.com/paper/2010.06595/) (2020)
- [A Critical Evaluation of Evaluations for Long-form Question Answering](https://mlsanity.com/paper/2305.18201/) (2023)
- [Do Massively Pretrained Language Models Make Better Storytellers?](https://mlsanity.com/paper/1909.10705/) (2019)

## Built on by (most cited first, 30 of 34)

- [LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs](https://mlsanity.com/paper/2408.07055/) (2024)
- [WritingBench: A Comprehensive Benchmark for Generative Writing](https://mlsanity.com/paper/2503.05244/) (2025)
- [HelloBench: Evaluating Long Text Generation Capabilities of Large Language Models](https://mlsanity.com/paper/2409.16191/) (2024)
- [CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraints](https://mlsanity.com/paper/2410.04197/) (2024)
- [Language Models can Self-Lengthen to Generate Long Texts](https://mlsanity.com/paper/2410.23933/) (2024)
- [RAPID: Efficient Retrieval-Augmented Long Text Generation with Writing Planning and Information Discovery](https://mlsanity.com/paper/2503.00751/) (2025)
- [OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinking](https://mlsanity.com/paper/2501.09751/) (2025)
- [SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generation](https://mlsanity.com/paper/2508.15658/) (2025)
- [LongEval: A Comprehensive Analysis of Long-Text Generation Through a Plan-based Paradigm](https://mlsanity.com/paper/2502.19103/) (2025)
- [Shifting Long-Context LLMs Research from Input to Output](https://mlsanity.com/paper/2503.04723/) (2025)
- [QUDsim: Quantifying Discourse Similarities in LLM-Generated Text](https://mlsanity.com/paper/2504.09373/) (2025)
- [LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning](https://mlsanity.com/paper/2506.18841/) (2025)
- [Contextualized Evaluations: Judging Language Model Responses to Underspecified Queries](https://mlsanity.com/paper/2411.07237/) (2024)
- [RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Data](https://mlsanity.com/paper/2505.19030/) (2025)
- [Constraint Back-translation Improves Complex Instruction Following of Large Language Models](https://mlsanity.com/paper/2410.24175/) (2024)
- [LongSafety: Enhance Safety for Long-Context LLMs](https://mlsanity.com/paper/2411.06899/) (2024)
- [CLIPPER: Compression enables long-context synthetic data generation](https://mlsanity.com/paper/2502.14854/) (2025)
- [AIR: Complex Instruction Generation via Automatic Iterative Refinement](https://mlsanity.com/paper/2502.17787/) (2025)
- [NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables](https://mlsanity.com/paper/2504.06560/) (2025)
- [Frankentext: Stitching random text fragments into long-form narratives](https://mlsanity.com/paper/2505.18128/) (2025)
- [Enhancing Human Evaluation in Machine Translation with Comparative Judgment](https://mlsanity.com/paper/2502.17797/) (2025)
- [AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism](https://mlsanity.com/paper/2506.03700/) (2025)
- [Lost in Stories: Consistency Bugs in Long Story Generation by LLMs](https://mlsanity.com/paper/2603.05890/) (2026)
- [SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models](https://mlsanity.com/paper/2506.04180/) (2025)
- [GateLens: A Reasoning-Enhanced LLM Agent for Automotive Software Release Analytics](https://mlsanity.com/paper/2503.21735/) (2025)
- [From General to Targeted Rewards: Surpassing GPT-4 in Open-Ended Long-Context Generation](https://mlsanity.com/paper/2506.16024/) (2025)
- [On Stable Long-Form Generation: Benchmarking and Mitigating Length Volatility](https://mlsanity.com/paper/2605.01357/) (2026)
- [Lost-in-the-Middle in Long-Text Generation: Synthetic Dataset, Evaluation Framework, and Mitigation](https://mlsanity.com/paper/2503.06868/) (2025)
- [LongWeave: A Long-Form Generation Benchmark Bridging Real-World Relevance and Verifiability](https://mlsanity.com/paper/2510.24345/) (2025)
- [Long-form RewardBench: Evaluating Reward Models for Long-form Generation](https://mlsanity.com/paper/2603.12963/) (2026)

## Abstract (arXiv)

Existing research on instruction following largely focuses on tasks with simple instructions and short responses. In this work, we explore multi-constraint instruction following for generating long-form text. We create Suri, a dataset with 20K human-written long-form texts paired with LLM-generated backtranslated instructions that contain multiple complex constraints. Because of prohibitive challenges associated with collecting human preference judgments on long-form texts, preference-tuning algorithms such as DPO are infeasible in our setting; thus, we propose Instructional ORPO (I-ORPO), an alignment method based on the ORPO algorithm. Instead of receiving negative feedback from dispreferred responses, I-ORPO obtains negative feedback from synthetically corrupted instructions generated by an LLM. Using Suri, we perform supervised and I-ORPO fine-tuning on Mistral-7b-Instruct-v0.2. The resulting models, Suri-SFT and Suri-I-ORPO, generate significantly longer texts (~5K tokens) than base models without significant quality deterioration. Our human evaluation shows that while both SFT and I-ORPO models satisfy most constraints, Suri-I-ORPO generations are generally preferred for their coherent and informative incorporation of the constraints. We release our code at https://github.com/chtmp223/suri.