mlsanityData to 7 Oct 2026

UnifiedQA: Crossing Format Boundaries With a Single QA System

arXiv 2005.00700 ↗First posted 2 May 2020cs.CL cs.AIcited by 475

Not on the map. Why?

Cited by 77 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Question answering (QA) tasks have been posed using a variety of formats, such as extractive span selection, multiple choice, etc. This has led to format-specialized models, and even to an implicit division in the QA community. We argue that such boundaries are artificial and perhaps unnecessary, given the reasoning abilities we seek to teach are not governed by the format. As evidence, we use the latest advances in language modeling to build a single pre-trained QA model, UnifiedQA, that performs surprisingly well across 17 QA datasets spanning 4 diverse formats. UnifiedQA performs on par with 9 different models that were trained on individual datasets themselves. Even when faced with 12 unseen datasets of observed formats, UnifiedQA performs surprisingly well, showing strong generalization from its out-of-format training data. Finally, simply fine-tuning this pre-trained QA model into specialized models results in a new state of the art on 6 datasets, establishing UnifiedQA as a strong starting point for building QA systems.

Key facts. Cited by 475 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 96; 2022: 132; 2023: 119; 2024: 75; 2025: 38; 2026: 15.

475 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  9. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  10. 2022Editing Models with Task Arithmeticcited by 1,076
  11. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  12. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
38 more
  1. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  2. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  3. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  4. 2022Galactica: A Large Language Model for Sciencecited by 736
  5. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  6. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  7. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  8. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  9. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  10. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  11. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  12. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  13. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  14. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  15. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  16. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  17. 2021MetaICL: Learning to Learn In Contextcited by 450
  18. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 416
  19. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  20. 2023Augmented Language Models: a Surveycited by 398
  21. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 386
  22. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  23. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  24. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  25. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  26. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 301
  27. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
  28. 2022UL2: Unifying Language Learning Paradigmscited by 292
  29. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  30. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
  31. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  32. 2022Transformer Memory as a Differentiable Search Indexcited by 242
  33. 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 241
  34. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
  35. 2021QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarizationcited by 233
  36. 2021PPT: Pre-trained Prompt Tuning for Few-shot Learningcited by 232
  37. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  38. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213

50 most cited of 475 All 475 (JSON)