mlsanityData to 7 Oct 2026

Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies

arXiv 2404.09022 ↗First posted 13 Apr 2024cs.LG cs.AI cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

With the surge of ChatGPT,the use of large models has significantly increased,rapidly rising to prominence across the industry and sweeping across the internet. This article is a comprehensive review of fine-tuning methods for large models. This paper investigates the latest technological advancements and the application of advanced methods in aspects such as task-adaptive fine-tuning,domain-adaptive fine-tuning,few-shot learning,knowledge distillation,multi-task learning,parameter-efficient fine-tuning,and dynamic fine-tuning.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 123 of its 159 references that are in the map. First posted on arXiv 13 Apr 2024.

Builds on

123 of 159 references.

References · 123

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  7. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  9. 2023GPT-4 Technical Reportcited by 18,132
  10. 2022Training language models to follow instructions with human feedbackcited by 15,712
  11. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  12. 2017Proximal Policy Optimization Algorithmscited by 14,441
38 more
  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  8. 2023Segment Anythingcited by 8,158
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  10. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  11. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  13. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  14. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  15. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  16. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  17. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  18. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  19. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  20. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  21. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  22. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  23. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  24. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  25. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  26. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  27. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  28. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  29. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  30. 2023Mistral 7Bcited by 3,065
  31. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  32. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  33. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  34. 2023A Survey of Large Language Modelscited by 2,537
  35. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  36. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  37. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  38. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081

50 most cited of 123 All 123 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2024Challenges in Adapting Multilingual LLMs to Low-Resource Languages using LoRA PEFT Tuningcited by 14
  2. 2026Gated Differentiable Working Memory for Long-Context Language Modelingcited by 6
  3. 2025Knowledge Graph-Infused Fine-Tuning for Structured Reasoning in Large Language Modelscited by 5
  4. 2025The potential -- and the pitfalls -- of using pre-trained language models as cognitive science theoriescited by 2
  5. 2026PermitGPT: A Unified Generative-AI Pipeline for Construction Hazard Forecasting, Permit Prediction, and Community Impactcited by 0