mlsanityData to 14 Oct 2026

ChatGPT in the Age of Generative AI and Large Language Models: A Concise Survey

arXiv 2307.04251 ↗First posted 9 Jul 2023cs.CL cs.AI cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

ChatGPT is a large language model (LLM) created by OpenAI that has been carefully trained on a large amount of data. It has revolutionized the field of natural language processing (NLP) and has pushed the boundaries of LLM capabilities. ChatGPT has played a pivotal role in enabling widespread public interaction with generative artificial intelligence (GAI) on a large scale. It has also sparked research interest in developing similar technologies and investigating their applications and implications. In this paper, our primary goal is to provide a concise survey on the current lines of research on ChatGPT and its evolution. We considered both the glass box and black box views of ChatGPT, encompassing the components and foundational elements of the technology, as well as its applications, impacts, and implications. The glass box approach focuses on understanding the inner workings of the technology, and the black box approach embraces it as a complex system, and thus examines its inputs, outputs, and effects. This paves the way for a comprehensive exploration of the technology and provides a road map for further research and experimentation. We also lay out essential foundational literature on LLMs and GAI in general and their connection with ChatGPT. This overview sheds light on existing and missing research lines in the emerging field of LLMs, benefiting both public users and developers. Furthermore, the paper delves into the broad spectrum of applications and significant concerns in fields such as education, research, healthcare, finance, etc.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 79 of its 226 references that are in the map. First posted on arXiv 9 Jul 2023.

Builds on

79 of 226 references.

References · 79

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2020Scaling Laws for Neural Language Modelscited by 6,586
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  10. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  11. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  12. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
38 more
  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  3. 2022Emergent Abilities of Large Language Modelscited by 2,512
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  7. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  8. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  9. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  10. 2021Variational Diffusion Modelscited by 1,153
  11. 2022Human Motion Diffusion Modelcited by 1,080
  12. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  13. 2023A Watermark for Large Language Modelscited by 725
  14. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  15. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  16. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  17. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 632
  18. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  19. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  20. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  21. 2020Jukebox: A Generative Model for Musiccited by 542
  22. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  23. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  24. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 481
  25. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
  26. 2023Can AI-Generated Text be Reliably Detected?cited by 386
  27. 2023Fine-Tuning Language Models with Just Forward Passescited by 345
  28. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 323
  29. 2022Teaching language models to support answers with verified quotescited by 292
  30. 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 268
  31. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 224
  32. 2021VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioningcited by 191
  33. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  34. 2022Talking About Large Language Modelscited by 175
  35. 2023A Categorical Archive of ChatGPT Failurescited by 168
  36. 2023Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speechcited by 166
  37. 2023Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERTcited by 155
  38. 2022Decoding speech perception from non-invasive brain recordingscited by 130

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 3; 2026: 1.

10 papers.