mlsanityData to 7 Oct 2026

Large Language Models Meet NLP: A Survey

arXiv 2405.12819 ↗First posted 21 May 2024cs.CL cs.AIcited by 102

Not on the map. Why?

Cited by 24 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

While large language models (LLMs) like ChatGPT have shown impressive capabilities in Natural Language Processing (NLP) tasks, a systematic investigation of their potential in this field remains largely unexplored. This study aims to address this gap by exploring the following questions: (1) How are LLMs currently applied to NLP tasks in the literature? (2) Have traditional NLP tasks already been solved with LLMs? (3) What is the future of the LLMs for NLP? To answer these questions, we take the first step to provide a comprehensive overview of LLMs in NLP. Specifically, we first introduce a unified taxonomy including (1) parameter-frozen paradigm and (2) parameter-tuning paradigm to offer a unified perspective for understanding the current progress of LLMs in NLP. Furthermore, we summarize the new frontiers and the corresponding challenges, aiming to inspire further groundbreaking advancements. We hope this work offers valuable insights into the potential and limitations of LLMs, while also serving as a practical guide for building effective LLMs in NLP.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). Builds on 205 of its 329 references that are in the map. First posted on arXiv 21 May 2024.

Builds on

205 of 329 references.

References · 205

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  12. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
38 more
  1. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  10. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  12. 2023A Survey of Large Language Modelscited by 2,537
  13. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  14. 2022Emergent Abilities of Large Language Modelscited by 2,483
  15. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  16. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  17. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  18. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  19. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  20. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  21. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  22. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  23. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  24. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  25. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  26. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  27. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  28. 2023StarCoder: may the source be with you!cited by 790
  29. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  30. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  31. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  32. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  33. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  34. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  35. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  36. 2022PAL: Program-aided Language Modelscited by 635
  37. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  38. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518

50 most cited of 205 All 205 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 21; 2025: 60; 2026: 21.

102 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2024Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thoughtcited by 56
  3. 2024What Factors Affect Multi-Modal In-Context Learning? An In-Depth Explorationcited by 31
  4. 2024CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Modelscited by 29
  5. 2025Text2World: Benchmarking Large Language Models for Symbolic World Model Generationcited by 25
  6. 2025Hallucination Detection in Large Language Models with Metamorphic Relationscited by 24
  7. 2025Youtu-GraphRAG: Vertically Unified Agents for Graph Retrieval-Augmented Complex Reasoningcited by 22
  8. 2025Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluationcited by 20
  9. 2024AutoCAP: Towards Automatic Cross-lingual Alignment Planning for Zero-shot Chain-of-Thoughtcited by 19
  10. 2025ViTCoT: Video-Text Interleaved Chain-of-Thought for Boosting Video Understanding in Large Language Modelscited by 18
  11. 2024Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Surveycited by 16
  12. 2025Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Modelscited by 16
38 more
  1. 2025Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems?cited by 15
  2. 2024What are the Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets? Insights and Best Practicescited by 14
  3. 2024Balancing Cost and Effectiveness of Synthetic Data Generation Strategies for LLMscited by 14
  4. 2025Recurrent Knowledge Identification and Fusion for Language Model Continual Learningcited by 14
  5. 2024CoPS: Empowering LLM Agents with Provable Cross-Task Experience Sharingcited by 12
  6. 2025From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMscited by 12
  7. 2025A Survey on Transformer Context Extension: Approaches and Evaluationcited by 11
  8. 2024Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Informationcited by 10
  9. 2025A Simple Review of EEG Foundation Models: Datasets, Advancements and Future Perspectivescited by 10
  10. 2025X-WebAgentBench: A Multilingual Interactive Web Benchmark for Evaluating Global Agentic Systemcited by 10
  11. 2024KIF: Knowledge Identification and Fusion for Language Model Continual Learningcited by 9
  12. 2025Comparative Analysis Based on DeepSeek, ChatGPT, and Google Gemini: Features, Techniques, Performance, Future Prospectscited by 9
  13. 2025Rethinking Tokenization for Rich Morphology: The Dominance of Unigram over BPE and Morphological Alignmentcited by 9
  14. 2024FLEXTAF: Enhancing Table Reasoning with Flexible Tabular Formatscited by 8
  15. 2025Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMscited by 8
  16. 2025Financial Named Entity Recognition: How Far Can LLM Go?cited by 7
  17. 2025DLPO: Towards a Robust, Efficient, and Generalizable Prompt Optimization Framework from a Deep-Learning Perspectivecited by 7
  18. 2025CCHall: A Novel Benchmark for Joint Cross-Lingual and Cross-Modal Hallucinations Detection in Large Language Modelscited by 7
  19. 2025Analyzing Political Bias in LLMs via Target-Oriented Sentiment Classificationcited by 7
  20. 2024NL-Eye: Abductive NLI for Imagescited by 5
  21. 2025Wrong Answers Can Also Be Useful: PlausibleQA -- A Large-Scale QA Dataset with Answer Plausibility Scorescited by 5
  22. 2025Small or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcarecited by 5
  23. 2025Adaptive LoRA Merge with Parameter Pruning for Low-Resource Generationcited by 5
  24. 2025SPARQL Query Generation with LLMs: Measuring the Impact of Training Data Memorization and Knowledge Injectioncited by 5
  25. 2024Dual-Phase Accelerated Prompt Optimizationcited by 4
  26. 2024WikiHint: A Human-Annotated Dataset for Hint Ranking and Generationcited by 4
  27. 2024LSAQ: Layer-Specific Adaptive Quantization for Large Language Model Deploymentcited by 4
  28. 2025Revitalizing Saturated Benchmarks: A Weighted Metric Approach for Differentiating Large Language Model Performancecited by 4
  29. 2025EMORL: Ensemble Multi-Objective Reinforcement Learning for Efficient and Flexible LLM Fine-Tuningcited by 4
  30. 2025Large Language Models and Arabic Content: A Reviewcited by 4
  31. 2025RBF++: Quantifying and Optimizing Reasoning Boundaries across Measurable and Unmeasurable Capabilities for Chain-of-Thought Reasoningcited by 4
  32. 2025R-Debater: Retrieval-Augmented Debate Generation through Argumentative Memorycited by 4
  33. 2026The Molecular Structure of Thought: Mapping the Topology of Long Chain-of-Thought Reasoningcited by 4
  34. 2026MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuningcited by 4
  35. 2024Raw Text is All you Need: Knowledge-intensive Multi-turn Instruction Tuning for Large Language Modelcited by 3
  36. 2025HintEval: An Open-Source Python Toolkit for Hint Generation and Hint Evaluationcited by 3
  37. 2025SPARC: Subspace-Aware Prompt Adaptation for Robust Continual Learning in LLMscited by 3
  38. 2025Cross-Format Retrieval-Augmented Generation in XR with LLMs for Context-Aware Maintenance Assistancecited by 3

50 most cited of 102 All 102 (JSON)