mlsanityData to 14 Oct 2026

Do Generative Large Language Models need billions of parameters?

arXiv 2309.06589 ↗First posted 12 Sep 2023cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

This paper presents novel systems and methodologies for the development of efficient large language models (LLMs). It explores the trade-offs between model size, performance, and computational resources, with the aim of maximizing the efficiency of these AI systems. The research explores novel methods that allow different parts of the model to share parameters, reducing the total number of unique parameters required. This approach ensures that the model remains compact without sacrificing its ability to learn and represent complex language structures. This study provides valuable insights and tools for creating more efficient and effective LLMs, contributing to a more sustainable and accessible future for AI language modeling.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 25 of its 54 references that are in the map. First posted on arXiv 12 Sep 2023.

Builds on

25 of 54 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  7. 2020Longformer: The Long-Document Transformercited by 3,413
  8. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  10. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  11. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  12. 2020Reformer: The Efficient Transformercited by 1,668
13 more
  1. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  2. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  3. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  4. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  5. 2019Text Summarization with Pretrained Encoderscited by 556
  6. 2013Do Deep Nets Really Need to be Deep?cited by 474
  7. 2020Towards a Human-like Open-Domain Chatbotcited by 470
  8. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 273
  9. 2018Deep Learning for Sentiment Analysis : A Surveycited by 139
  10. 2018Troubling Trends in Machine Learning Scholarshipcited by 108
  11. 2020O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformerscited by 83
  12. 2021Zero-Shot Open-Book Question Answeringcited by 6
  13. 2022Alexa, Predict My Flight Delaycited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 1; 2025: 3; 2026: 1.

8 papers.

  1. 2025Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computationcited by 133
  2. 2023Does Synthetic Data Make Large Language Models More Efficient?cited by 8
  3. 2023Can pruning make Large Language Models more efficient?cited by 5
  4. 2023Can a student Large Language Model perform as well as it's teacher?cited by 4
  5. 2026MoRE: Mixture of Reused Expertscited by 3
  6. 2024A critical review of methods and challenges in large language modelscited by 2
  7. 2025Achieving Trustworthy Real-Time Decision Support Systems with Low-Latency Interpretable AI Modelscited by 1
  8. 2025Investigating Recent Large Language Models for Vietnamese Machine Reading Comprehensioncited by 0