mlsanityData to 7 Oct 2026

A Survey on Large Language Models from Concept to Implementation

arXiv 2403.18969 ↗First posted 27 Mar 2024cs.CL cs.AI cs.ITcited by 4

Not on the map. Why?

Abstract (arXiv)

Recent advancements in Large Language Models (LLMs), particularly those built on Transformer architectures, have significantly broadened the scope of natural language processing (NLP) applications, transcending their initial use in chatbot technology. This paper investigates the multifaceted applications of these models, with an emphasis on the GPT series. This exploration focuses on the transformative impact of artificial intelligence (AI) driven tools in revolutionizing traditional tasks like coding and problem-solving, while also paving new paths in research and development across diverse industries. From code interpretation and image captioning to facilitating the construction of interactive systems and advancing computational domains, Transformer models exemplify a synergy of deep learning, data analysis, and neural network design. This survey provides an in-depth look at the latest research in Transformer models, highlighting their versatility and the potential they hold for transforming diverse application sectors, thereby offering readers a comprehensive understanding of the current and future landscape of Transformer-based LLMs in practical applications.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 71 of its 122 references that are in the map. First posted on arXiv 27 Mar 2024.

Builds on

71 of 122 references.

References · 71

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  10. 2021Program Synthesis with Large Language Modelscited by 3,509
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  12. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
38 more
  1. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  2. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  3. 2023PaLM 2 Technical Reportcited by 1,154
  4. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  5. 2021Transformers in Vision: A Surveycited by 1,009
  6. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  7. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  8. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  9. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  10. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  11. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  12. 2022Using cognitive psychology to understand GPT-3cited by 389
  13. 2021Pay Attention to MLPscited by 367
  14. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  15. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  16. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
  17. 2019Attention on Attention for Image Captioningcited by 289
  18. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 279
  19. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 215
  20. 2021A Survey of Human-in-the-loop for Machine Learningcited by 214
  21. 2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196
  22. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  23. 2020Few-shot Image Generation with Elastic Weight Consolidationcited by 122
  24. 2023Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!cited by 119
  25. 2022Chinese CLIP: Contrastive Vision-Language Pretraining in Chinesecited by 108
  26. 2022Text-Only Training for Image Captioning using Noise-Injected CLIPcited by 91
  27. 2023DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Trainingcited by 84
  28. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  29. 2022SmallCap: Lightweight Image Captioning Prompted with Retrieval Augmentationcited by 82
  30. 2015Zero-Shot Learning via Semantic Similarity Embeddingcited by 74
  31. 2018NRTR: A No-Recurrence Sequence-to-Sequence Model For Scene Text Recognitioncited by 73
  32. 2022Repair Is Nearly Generation: Multilingual Program Repair with LLMscited by 68
  33. 2023ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposalscited by 47
  34. 2023Exploring Large Language Model for Graph Data Understanding in Online Job Recommendationscited by 45
  35. 2022ProposalCLIP: Unsupervised Open-Category Object Proposal Generation via Exploiting CLIP Cuescited by 42
  36. 2022Multi-View Document Representation Learning for Open-Domain Dense Retrievalcited by 41
  37. 2017Deep Reinforcement Learning-based Image Captioning with Embedding Rewardcited by 39
  38. 2023CodeFusion: A Pre-trained Diffusion Model for Code Generationcited by 31

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 2.

4 papers.

  1. 2024Pixels to Prose: Understanding the art of Image Captioningcited by 1
  2. 2025A Case Study Investigating the Role of Generative AI in Quality Evaluations of Epics in Agile Software Developmentcited by 1
  3. 2026Are Multimodal LLMs Ready for Surveillance? A Reality Check on Zero-Shot Anomaly Detection in the Wildcited by 1
  4. 2026When Prompts Become Payloads: A Framework for Mitigating SQL Injection Attacks in Large Language Model-Driven Applicationscited by 0