mlsanityData to 14 Oct 2026

Enhancing Chat Language Models by Scaling High-quality Instructional Conversations

arXiv 2305.14233 ↗First posted 23 May 2023cs.CL cs.AIcited by 839

Not on the map. Why?

Cited by 192 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Fine-tuning on instruction data has been widely validated as an effective practice for implementing chat language models like ChatGPT. Scaling the diversity and quality of such data, although straightforward, stands a great chance of leading to improved performance. This paper aims to improve the upper bound of open-source models further. We first provide a systematically designed, diverse, informative, large-scale dataset of instructional conversations, UltraChat, which does not involve human queries. Our objective is to capture the breadth of interactions that a human might have with an AI assistant and employs a comprehensive framework to generate multi-turn conversation iteratively. UltraChat contains 1.5 million high-quality multi-turn dialogues and covers a wide range of topics and instructions. Our statistical analysis of UltraChat reveals its superiority in various key metrics, including scale, average length, diversity, coherence, etc., solidifying its position as a leading open-source dataset. Building upon UltraChat, we fine-tune a LLaMA model to create a powerful conversational model, UltraLLaMA. Our evaluations indicate that UltraLLaMA consistently outperforms other open-source models, including Vicuna, the previously recognized state-of-the-art open-source model. The dataset and the model will be publicly releasedfootnoteurlhttps://github.com/thunlp/UltraChat.

Key facts. Cited by 839 papers in the map (data through 14 Oct 2026). Builds on 26 of its 43 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

26 of 43 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
14 more
  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  3. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  4. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  5. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  6. 2023LIMA: Less Is More for Alignmentcited by 1,100
  7. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  8. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  9. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  10. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  11. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  12. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  13. 2022SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualizationcited by 168
  14. 2023Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Casescited by 86

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 53; 2024: 308; 2025: 282; 2026: 196.

839 papers.

Sort:

By area:

  1. 2023Qwen Technical Reportcited by 3,344
  2. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  5. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  6. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  7. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  8. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  9. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  10. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  11. 2024Yi: Open Foundation Models by 01.AIcited by 719
  12. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
38 more
  1. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  2. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  3. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  4. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
  5. 2023Zephyr: Direct Distillation of LM Alignmentcited by 506
  6. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  7. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 461
  8. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  9. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 406
  10. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 364
  11. 2024Alignment faking in large language modelscited by 360
  12. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  13. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 331
  14. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
  15. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 310
  16. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
  17. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  18. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  19. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 257
  20. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 252
  21. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 250
  22. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  23. 2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 236
  24. 2024Generative Representational Instruction Tuningcited by 233
  25. 2024Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agentscited by 218
  26. 2024When Attention Sink Emerges in Language Models: An Empirical Viewcited by 208
  27. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  28. 2024World Model on Million-Length Video And Language With Blockwise RingAttentioncited by 198
  29. 2024ReFT: Representation Finetuning for Language Modelscited by 197
  30. 2024Advancing LLM Reasoning Generalists with Preference Treescited by 192
  31. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 187
  32. 2024Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Datacited by 185
  33. 2026Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolationcited by 184
  34. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencescited by 178
  35. 2023RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Modelscited by 175
  36. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 173
  37. 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 173
  38. 2024Free Process Rewards without Process Labelscited by 165

50 most cited of 839 All 839 (JSON)