mlsanityData to 14 Oct 2026

Several categories of Large Language Models (LLMs): A Short Survey

arXiv 2307.10188 ↗First posted 5 Jul 2023cs.CL cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Large Language Models(LLMs)have become effective tools for natural language processing and have been used in many different fields. This essay offers a succinct summary of various LLM subcategories. The survey emphasizes recent developments and efforts made for various LLM kinds, including task-based financial LLMs, multilingual language LLMs, biomedical and clinical LLMs, vision language LLMs, and code language models. The survey gives a general summary of the methods, attributes, datasets, transformer models, and comparison metrics applied in each category of LLMs. Furthermore, it highlights unresolved problems in the field of developing chatbots and virtual assistants, such as boosting natural language processing, enhancing chatbot intelligence, and resolving moral and legal dilemmas. The purpose of this study is to provide readers, developers, academics, and users interested in LLM-based chatbots and virtual intelligent assistant technologies with useful information and future directions.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 87 of its 138 references that are in the map. First posted on arXiv 5 Jul 2023.

Builds on

87 of 138 references.

References · 87

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  10. 2021Program Synthesis with Large Language Modelscited by 3,637
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  12. 2022Training Compute-Optimal Large Language Modelscited by 3,019
38 more
  1. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  2. 2023A Survey of Large Language Modelscited by 2,576
  3. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  5. 2018Deep contextualized word representationscited by 2,290
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  7. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  8. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  9. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  10. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  11. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  12. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  13. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  14. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  15. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  16. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  17. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  18. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  19. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  20. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  21. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  22. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  23. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  24. 2023BloombergGPT: A Large Language Model for Financecited by 939
  25. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  26. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  27. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  28. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  29. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  30. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  31. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  32. 2023Instruction Tuning with GPT-4cited by 686
  33. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
  34. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  35. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 597
  36. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  37. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  38. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529

50 most cited of 87 All 87 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 1; 2026: 1.

7 papers.