mlsanityData to 7 Oct 2026

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

arXiv 1804.07461 ↗First posted 20 Apr 2018cs.CLcited by 4,854

Not on the map. Why?

Cited by 130 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

For natural language understanding (NLU) technology to be maximally useful, both practically and as a scientific object of study, it must be general: it must be able to process language in a way that is not exclusively tailored to any one specific task or dataset. In pursuit of this objective, we introduce the General Language Understanding Evaluation benchmark (GLUE), a tool for evaluating and analyzing the performance of models across a diverse range of existing NLU tasks. GLUE is model-agnostic, but it incentivizes sharing knowledge across tasks because certain tasks have very limited training data. We further provide a hand-crafted diagnostic test suite that enables detailed linguistic analysis of NLU models. We evaluate baselines based on current methods for multi-task and transfer learning and find that they do not immediately give substantial improvements over the aggregate performance of training a separate model per task, indicating room for improvement in developing general and robust NLU systems.

Key facts. Cited by 4,854 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 707; 2022: 811; 2023: 964; 2024: 1002; 2025: 858; 2026: 510.

4,854 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  9. 2023A Survey of Large Language Modelscited by 2,537
  10. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  11. 2022Visual Prompt Tuningcited by 1,674
  12. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
38 more
  1. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  2. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  3. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  4. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  5. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  6. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  7. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  8. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  10. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  11. 2022Editing Models with Task Arithmeticcited by 1,076
  12. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  13. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  14. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  15. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  16. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  17. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  18. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  19. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  20. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  21. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  22. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  23. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  24. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  25. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  26. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  27. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  28. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  29. 2021GPT Understands, Toocited by 715
  30. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  31. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  32. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  33. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  34. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  35. 2022Large Language Models Can Self-Improvecited by 656
  36. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  37. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  38. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645

50 most cited of 4,854 All 4,854 (JSON)