# Prompting Frameworks for Large Language Models: A Survey

arXiv: https://arxiv.org/abs/2311.12785 · HTML: https://mlsanity.com/paper/2311.12785/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2023.

## Built on by (most cited first, 29 of 29)

- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges](https://mlsanity.com/paper/2505.10468/) (2025)
- [CogBench: a large language model walks into a psychology lab](https://mlsanity.com/paper/2402.18225/) (2024)
- [Human-Imperceptible Retrieval Poisoning Attacks in LLM-Powered Applications](https://mlsanity.com/paper/2404.17196/) (2024)
- [CrisisSense-LLM: Instruction Fine-Tuned Large Language Model for Multi-label Social Media Text Classification in Disaster Informatics](https://mlsanity.com/paper/2406.15477/) (2024)
- [Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoning](https://mlsanity.com/paper/2502.15361/) (2025)
- [Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation](https://mlsanity.com/paper/2404.06910/) (2024)
- [PDL: A Declarative Prompt Programming Language](https://mlsanity.com/paper/2410.19135/) (2024)
- [Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics](https://mlsanity.com/paper/2505.18658/) (2025)
- [Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective](https://mlsanity.com/paper/2412.12276/) (2024)
- [Towards Automating Text Annotation: A Case Study on Semantic Proximity Annotation using GPT-4](https://mlsanity.com/paper/2407.04130/) (2024)
- [LLM Access Shield: Domain-Specific LLM Framework for Privacy Policy Compliance](https://mlsanity.com/paper/2505.17145/) (2025)
- [Safeguarding Decentralized Social Media: LLM Agents for Automating Community Rule Compliance](https://mlsanity.com/paper/2409.08963/) (2024)
- [Towards Goal-oriented Prompt Engineering for Large Language Models: A Survey](https://mlsanity.com/paper/2401.14043/) (2024)
- [Advancing Responsible Innovation in Agentic AI: A study of Ethical Frameworks for Household Automation](https://mlsanity.com/paper/2507.15901/) (2025)
- [PATENTWRITER: A Benchmarking Study for Patent Drafting with LLMs](https://mlsanity.com/paper/2507.22387/) (2025)
- [Disentangling Interaction and Bias Effects in Opinion Dynamics of Large Language Models](https://mlsanity.com/paper/2509.06858/) (2025)
- [RE-GAINS & EnChAnT: Intelligent Tool Manipulation Systems For Enhanced Query Responses](https://mlsanity.com/paper/2401.15724/) (2024)
- [Talking the Talk Does Not Entail Walking the Walk: On the Limits of Large Language Models in Lexical Entailment Recognition](https://mlsanity.com/paper/2406.14894/) (2024)
- [KG-CF: Knowledge Graph Completion with Context Filtering under the Guidance of Large Language Models](https://mlsanity.com/paper/2501.02711/) (2025)
- [Audio Visual Segmentation Through Text Embeddings](https://mlsanity.com/paper/2502.16359/) (2025)
- [Prompt Engineering for Scale Development in Generative Psychometrics](https://mlsanity.com/paper/2603.15909/) (2026)
- ["Moralized" Multi-Step Jailbreak Prompts: Black-Box Testing of Guardrails in Large Language Models for Verbal Attacks](https://mlsanity.com/paper/2411.16730/) (2024)
- [EduBot -- Can LLMs Solve Personalized Learning and Programming Assignments?](https://mlsanity.com/paper/2504.17824/) (2025)
- [Instruction Learning Paradigms: A Dual Perspective on White-box and Black-box LLMs](https://mlsanity.com/paper/2506.21573/) (2025)
- [Forecasting Future Language: Context Design for Mention Markets](https://mlsanity.com/paper/2602.21229/) (2026)
- [From Detection to Response: A Deep Learning and Retrieval-Augmented Generation Framework for Network Intrusion Mitigation](https://mlsanity.com/paper/2605.17960/) (2026)
- [IstGPT: LLM-based Anomaly Detection for Spatial-Temporal Graph in Industrial Systems](https://mlsanity.com/paper/2606.01691/) (2026)
- [Grading the Graders: Verification Autonomy Levels (L0-L5) for LLM Reasoning](https://mlsanity.com/paper/2608.19009/) (2026)

## Abstract (arXiv)

Since the launch of ChatGPT, a powerful AI Chatbot developed by OpenAI, large language models (LLMs) have made significant advancements in both academia and industry, bringing about a fundamental engineering paradigm shift in many areas. While LLMs are powerful, it is also crucial to best use their power where "prompt'' plays a core role. However, the booming LLMs themselves, including excellent APIs like ChatGPT, have several inherent limitations: 1) temporal lag of training data, and 2) the lack of physical capabilities to perform external actions. Recently, we have observed the trend of utilizing prompt-based tools to better utilize the power of LLMs for downstream tasks, but a lack of systematic literature and standardized terminology, partly due to the rapid evolution of this field. Therefore, in this work, we survey related prompting tools and promote the concept of the "Prompting Framework" (PF), i.e. the framework for managing, simplifying, and facilitating interaction with large language models. We define the lifecycle of the PF as a hierarchical structure, from bottom to top, namely: Data Level, Base Level, Execute Level, and Service Level. We also systematically depict the overall landscape of the emerging PF field and discuss potential future research and challenges. To continuously track the developments in this area, we maintain a repository at https://github.com/lxx0628/Prompting-Framework-Survey, which can be a useful resource sharing platform for both academic and industry in this field.