mlsanityData to 7 Oct 2026

Efficient Prompting Methods for Large Language Models: A Survey

arXiv 2404.01077 ↗First posted 1 Apr 2024cs.CLcited by 46

Not on the map. Why?

Abstract (arXiv)

Prompting is a mainstream paradigm for adapting large language models to specific natural language processing tasks without modifying internal parameters. Therefore, detailed supplementary knowledge needs to be integrated into external prompts, which inevitably brings extra human efforts and computational burdens for practical applications. As an effective solution to mitigate resource consumption, Efficient Prompting Methods have attracted a wide range of attention. We provide mathematical expressions at a high level to deeply discuss Automatic Prompt Engineering for different prompt components and Prompt Compression in continuous and discrete spaces. Finally, we highlight promising future directions to inspire researchers interested in this field.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 108 of its 144 references that are in the map. First posted on arXiv 1 Apr 2024.

Builds on

108 of 144 references.

References · 108

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2017Proximal Policy Optimization Algorithmscited by 14,441
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  10. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  11. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  12. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
38 more
  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  10. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  13. 2022Emergent Abilities of Large Language Modelscited by 2,483
  14. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  15. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  16. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  17. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  18. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  19. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  20. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  21. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  22. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  23. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  24. 2019How Can We Know What Language Models Know?cited by 953
  25. 2023Large Language Models as Optimizerscited by 830
  26. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  27. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  28. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  29. 2022Large Language Models Can Self-Improvecited by 656
  30. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  31. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  32. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  33. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  34. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
  35. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  36. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  37. 2023Adapting Language Models to Compress Contextscited by 350
  38. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347

50 most cited of 108 All 108 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 16; 2026: 13.

46 papers.

Sort:

By area:

  1. 2024Prompt Compression for Large Language Models: A Surveycited by 65
  2. 2025Self-Supervised Prompt Optimizationcited by 37
  3. 2024Probing the limitations of multimodal language models for chemistry and materials researchcited by 31
  4. 2024Bias patterns in the application of LLMs for clinical decision support: A comprehensive studycited by 27
  5. 2025SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignmentcited by 17
  6. 2025Improved Representation Steering for Language Modelscited by 17
  7. 2025Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchorscited by 14
  8. 2025ProAPO: Progressively Automatic Prompt Optimization for Visual Classificationcited by 12
  9. 2024Implicit In-context Learningcited by 11
  10. 2024Large Language Models Know What To Say But Not When To Speakcited by 11
  11. 2024MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understandingcited by 10
  12. 2025Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translationcited by 8
34 more
  1. 2025Prompt-MII: Meta-Learning Instruction Induction for LLMscited by 8
  2. 2024Aligning (Medical) LLMs for (Counterfactual) Fairnesscited by 7
  3. 2025Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retentioncited by 7
  4. 2024Navigating the Landscape of Hint Generation Research: From the Past to the Futurecited by 6
  5. 2025A Modular Taxonomy for Hate Speech Definitions and Its Impact on Zero-Shot LLM Classification Performancecited by 6
  6. 2025Assessing Coherency and Consistency of Code Execution Reasoning by Large Language Modelscited by 5
  7. 2024LRHP: Learning Representations for Human Preferences via Preference Pairscited by 4
  8. 2026Enhancing Instruction Following of LLMs via Activation Steering with Dynamic Rejectioncited by 4
  9. 2026MAS-PromptBench: When Does Prompt Optimization Improve Multi-Agent LLM Systems?cited by 4
  10. 2024Topic Classification of Case Law Using a Large Language Model and a New Taxonomy for UK Law: AI Insights into Summary Judgmentcited by 3
  11. 2024When Not to Answer: Evaluating Prompts on GPT Models for Effective Abstention in Unanswerable Math Word Problemscited by 2
  12. 2024From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmapcited by 2
  13. 2024The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcarecited by 2
  14. 2024Knowledge Graphs are all you need: Leveraging KGs in Physics Question Answeringcited by 2
  15. 2025When Dimensionality Hurts: The Role of LLM Embedding Compression for Noisy Regression Taskscited by 2
  16. 2025PAFT: Prompt-Agnostic Fine-Tuningcited by 2
  17. 2025Rumor Detection by Multi-task Suffix Learning based on Time-series Dual Sentimentscited by 2
  18. 2025Balancing Content Size in RAG-Text2SQL Systemcited by 2
  19. 2025MoT: Modularization-of-Thought Prompting for Effective Code Generationcited by 2
  20. 2024Open Artificial Knowledgecited by 1
  21. 2024Pairing Analogy-Augmented Generation with Procedural Memory for Procedural Q&Acited by 1
  22. 2026MemoSight: Unifying Context Compression and Multi Token Prediction for Reasoning Accelerationcited by 1
  23. 2026How to Ask the AI: A User Perspective Survey for Large Language Model Promptingcited by 1
  24. 2024Eliciting Causal Abilities in Large Language Models for Reasoning Taskscited by 0
  25. 2025SI-Agent: An Agentic Framework for Feedback-Driven Generation and Tuning of Human-Readable System Instructions for Large Language Modelscited by 0
  26. 2026APR: Penalizing Structural Redundancy in Large Reasoning Models via Anchor-based Process Rewardscited by 0
  27. 2026Language Model Inversion through End-to-End Differentiationcited by 0
  28. 2026Optimizing Soft Prompt Tuning via Structural Evolutioncited by 0
  29. 2026When Scaling Fails: Mitigating Audio Perception Decay of LALMs via Multi-Step Perception-Aware Reasoningcited by 0
  30. 2026Towards a Data-Parameter Correspondence for LLMs: A Preliminary Discussioncited by 0
  31. 2026PragLocker: Protecting Agent Intellectual Property in Untrusted Deployments via Non-Portable Promptscited by 0
  32. 2026HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Stepscited by 0
  33. 2026Automatic Model Card Generation Using an LLMcited by 0
  34. 2026Large language models in medical time series analysiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
  2. 2025An Empirical Study on Prompt Compression for Large Language Models
  3. 2025Understanding and Improving Information Preservation in Prompt Compression for LLMs
  4. 2026Lexical Prompt Compression for Large Language Models: A Training-Free, Deterministic Pipeline with Empirical Pareto Analysis Across Eleven Task Categories
  5. 2026Prompt Minimization: Reducing Input Redundancy Without Sacrificing Output Fidelity
  6. 2024Towards Generalist Prompting for Large Language Models by Mental Models
  7. 2025Dynamic Compressing Prompts for Efficient Inference of Large Language Models
  8. 2024Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models