# Unleashing the potential of prompt engineering for large language models

arXiv: https://arxiv.org/abs/2310.14735 · HTML: https://mlsanity.com/paper/2310.14735/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- Builds on 219 of its 302 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Builds on (most cited first, 30 of 219)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)

## Built on by (most cited first, 30 of 72)

- [Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents](https://mlsanity.com/paper/2505.22954/) (2025)
- [OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking System](https://mlsanity.com/paper/2509.18091/) (2025)
- [SmileyLlama: Modifying Large Language Models for Directed Chemical Space Exploration](https://mlsanity.com/paper/2409.02231/) (2024)
- [Benchmarking Foundation Models for Zero-Shot Biometric Tasks](https://mlsanity.com/paper/2505.24214/) (2025)
- [MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks](https://mlsanity.com/paper/2507.19634/) (2025)
- [LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Language](https://mlsanity.com/paper/2402.16929/) (2024)
- [Motion meets Attention: Video Motion Prompts](https://mlsanity.com/paper/2407.03179/) (2024)
- [Promptomatix: An Automatic Prompt Optimization Framework for Large Language Models](https://mlsanity.com/paper/2507.14241/) (2025)
- [Metamorphic Testing of Large Language Models for Natural Language Processing](https://mlsanity.com/paper/2511.02108/) (2025)
- [Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Products](https://mlsanity.com/paper/2410.12071/) (2024)
- [Failure Modes in LLM Systems: A System-Level Taxonomy for Reliable AI Applications](https://mlsanity.com/paper/2511.19933/) (2025)
- [NTSEBENCH: Cognitive Reasoning Benchmark for Vision Language Models](https://mlsanity.com/paper/2407.10380/) (2024)
- [Computational Hermeneutics: Evaluating generative AI as a cultural technology](https://mlsanity.com/paper/2604.16403/) (2026)
- [RoBERTa-BiLSTM: A Context-Aware Hybrid Model for Sentiment Analysis](https://mlsanity.com/paper/2406.00367/) (2024)
- [Multilingual Prompt Engineering in Large Language Models: A Survey Across NLP Tasks](https://mlsanity.com/paper/2505.11665/) (2025)
- [Large Language Models and Evolutionary Computation: A Critical Review of Bidirectional Interaction, Automated Algorithm Design, and Co-Adaptive Systems](https://mlsanity.com/paper/2505.15741/) (2025)
- [Impact of Decentralized Learning on Player Utilities in Stackelberg Games](https://mlsanity.com/paper/2403.00188/) (2024)
- [Improved IR-based Bug Localization with Intelligent Relevance Feedback](https://mlsanity.com/paper/2501.10542/) (2025)
- [LLM-Agents Driven Automated Simulation Testing and Analysis of small Uncrewed Aerial Systems](https://mlsanity.com/paper/2501.11864/) (2025)
- [RAG-Pull: Turning Retrieval into a Code-Injection Channel via Invisible Unicode Perturbations](https://mlsanity.com/paper/2510.11195/) (2025)
- [A Course Intelligence Platform for Higher Education: Lessons from AI-Assisted Course Evaluation](https://mlsanity.com/paper/2411.02455/) (2024)
- [Trace-of-Thought Prompting: Investigating Prompt-Based Knowledge Distillation Through Question Decomposition](https://mlsanity.com/paper/2504.20946/) (2025)
- [Telco-oRAG: Optimizing Retrieval-augmented Generation for Telecom Queries via Hybrid Retrieval and Neural Routing](https://mlsanity.com/paper/2505.11856/) (2025)
- [Prompt to Protection: A Comparative Study of Multimodal LLMs in Construction Hazard Recognition](https://mlsanity.com/paper/2506.07436/) (2025)
- [Textual interpretation of transient image classifications from large language models](https://mlsanity.com/paper/2510.06931/) (2025)
- [VIP-COP: Context Optimization for Tabular Foundation Models](https://mlsanity.com/paper/2605.12904/) (2026)
- [OnlySportsLM: Optimizing Sports-Domain Language Models with SOTA Performance under Billion Parameters](https://mlsanity.com/paper/2409.00286/) (2024)
- [iToT: An Interactive System for Customized Tree-of-Thought Generation](https://mlsanity.com/paper/2409.00413/) (2024)
- [One Patient, Many Contexts: Scaling Medical AI with Contextual Intelligence](https://mlsanity.com/paper/2506.10157/) (2025)
- [Evaluating Large Language Models (LLMs) in Financial NLP: A Comparative Study on Financial Report Analysis](https://mlsanity.com/paper/2507.22936/) (2025)

## Abstract (arXiv)

This comprehensive review delves into the pivotal role of prompt engineering in unleashing the capabilities of Large Language Models (LLMs). The development of Artificial Intelligence (AI), from its inception in the 1950s to the emergence of advanced neural networks and deep learning architectures, has made a breakthrough in LLMs, with models such as GPT-4o and Claude-3, and in Vision-Language Models (VLMs), with models such as CLIP and ALIGN. Prompt engineering is the process of structuring inputs, which has emerged as a crucial technique to maximize the utility and accuracy of these models. This paper explores both foundational and advanced methodologies of prompt engineering, including techniques such as self-consistency, chain-of-thought, and generated knowledge, which significantly enhance model performance. Additionally, it examines the prompt method of VLMs through innovative approaches such as Context Optimization (CoOp), Conditional Context Optimization (CoCoOp), and Multimodal Prompt Learning (MaPLe). Critical to this discussion is the aspect of AI security, particularly adversarial attacks that exploit vulnerabilities in prompt engineering. Strategies to mitigate these risks and enhance model robustness are thoroughly reviewed. The evaluation of prompt methods is also addressed through both subjective and objective metrics, ensuring a robust analysis of their efficacy. This review also reflects the essential role of prompt engineering in advancing AI capabilities, providing a structured framework for future research and application.