mlsanityData to 7 Oct 2026

Parrot: Enhancing Multi-Turn Instruction Following for Large Language Models

arXiv 2310.07301 ↗First posted 11 Oct 2023cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Humans often interact with large language models (LLMs) in multi-turn interaction to obtain desired answers or more information. However, most existing studies overlook the multi-turn instruction following ability of LLMs, in terms of training dataset, training method, and evaluation benchmark. In this paper, we introduce Parrot, a solution aiming to enhance multi-turn instruction following for LLMs. First, we introduce an efficient but effective method for collecting multi-turn instructions that feature human-like queries, such as anaphora and ellipsis. Second, we propose a context-aware preference optimization strategy to further enhance LLMs for complex queries in multi-turn interaction. Moreover, to quantitatively evaluate LLMs in multi-turn instruction following, we manually build a multi-turn benchmark derived from existing ones. Extensive experiments show that Parrot improves current LLMs by up to 7.2% in multi-turn instruction following. Our dataset and codes will be open-sourced to facilitate future research.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 16 of its 27 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

16 of 27 references.

References · 16

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2020Learning to summarize from human feedbackcited by 2,562
  9. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  10. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  11. 2023LIMA: Less Is More for Alignmentcited by 1,080
  12. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
4 more
  1. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  2. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  3. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  4. 2023Can Large Language Models Understand Real-World Complex Instructions?cited by 69

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 6; 2026: 3.

15 papers.

Sort:

By area:

  1. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  2. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 247
  3. 2024Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Trainingcited by 53
  4. 2024FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMscited by 36
  5. 2024MMMT-IF: A Challenging Multimodal Multi-Turn Instruction Following Benchmarkcited by 8
  6. 2024MORTAR: Multi-turn Metamorphic Testing for LLM-based Dialogue Systemscited by 7
  7. 2024ExGra-Med: Extended Context Graph Alignment for Medical Vision-Language Modelscited by 6
  8. 2026Knowledge-Driven Multi-Turn Jailbreaking on Large Language Modelscited by 6
  9. 2025MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluationcited by 4
  10. 2025DynaKV: Enabling Accurate and Efficient Long-Sequence LLM Decoding on Smartphonescited by 4
  11. 2026Benchmarking Multi-turn Medical Diagnosis: Hold, Lure, and Self-Correctioncited by 4
  12. 2025Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs' Instruction Following Capabilitycited by 3
3 more
  1. 2025Test-Time Policy Adaptation for Enhanced Multi-Turn Interactions with LLMscited by 3
  2. 2026Implicit Turn-Wise Policy Optimization for Proactive User-LLM Interactioncited by 3
  3. 2025KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialoguescited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Multi-Turn Interaction Capabilities of Large Language Models
  2. 2023FollowEval: A Multi-Dimensional Benchmark for Assessing the Instruction-Following Capability of Large Language Models
  3. 2024Mixture-of-Instructions: Aligning Large Language Models via Mixture Prompting
  4. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback
  5. 2026Beyond Continuity: Challenges of Context Switching in Multi-Turn Dialogue with LLMs
  6. 2023BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Models
  7. 2024Diverse and Fine-Grained Instruction-Following Ability Exploration with Synthetic Data
  8. 2025Can Language Models Follow Multiple Turns of Entangled Instructions?