# TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Tasks

arXiv: https://arxiv.org/abs/2305.11430 · HTML: https://mlsanity.com/paper/2305.11430/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 19 May 2023.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [STaR: Bootstrapping Reasoning With Reasoning](https://mlsanity.com/paper/2203.14465/) (2022)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://mlsanity.com/paper/2201.11990/) (2022)
- [Decomposed Prompting: A Modular Approach for Solving Complex Tasks](https://mlsanity.com/paper/2210.02406/) (2022)
- [Complexity-Based Prompting for Multi-Step Reasoning](https://mlsanity.com/paper/2210.00720/) (2022)
- [Language Models can Solve Computer Tasks](https://mlsanity.com/paper/2303.17491/) (2023)
- [Explain Yourself! Leveraging Language Models for Commonsense Reasoning](https://mlsanity.com/paper/1906.02361/) (2019)
- [PTR: Prompt Tuning with Rules for Text Classification](https://mlsanity.com/paper/2105.11259/) (2021)
- [What Does it Mean for a Language Model to Preserve Privacy?](https://mlsanity.com/paper/2202.05520/) (2022)
- [Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango](https://mlsanity.com/paper/2209.07686/) (2022)
- [MReD: A Meta-Review Dataset for Structure-Controllable Text Generation](https://mlsanity.com/paper/2110.07474/) (2021)
- [A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization](https://mlsanity.com/paper/2305.08503/) (2023)

## Built on by (most cited first, 30 of 49)

- [The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models](https://mlsanity.com/paper/2401.03205/) (2024)
- [Determinants of LLM-assisted Decision-Making](https://mlsanity.com/paper/2402.17385/) (2024)
- [UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All](https://mlsanity.com/paper/2403.12532/) (2024)
- [Argumentative Large Language Models for Explainable and Contestable Claim Verification](https://mlsanity.com/paper/2405.02079/) (2024)
- [Is GPT-4 a reliable rater? Evaluating Consistency in GPT-4 Text Ratings](https://mlsanity.com/paper/2308.02575/) (2023)
- [Balancing Autonomy and Alignment: A Multi-Dimensional Taxonomy for Autonomous LLM-powered Multi-Agent Architectures](https://mlsanity.com/paper/2310.03659/) (2023)
- [iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvement](https://mlsanity.com/paper/2407.06025/) (2024)
- [ChatGPT as your Personal Data Scientist](https://mlsanity.com/paper/2305.13657/) (2023)
- [Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applications](https://mlsanity.com/paper/2406.10300/) (2024)
- [LLMs as Meta-Reviewers' Assistants: A Case Study](https://mlsanity.com/paper/2402.15589/) (2024)
- [Taxonomy of User Needs and Actions](https://mlsanity.com/paper/2510.06124/) (2025)
- [Systematic Task Exploration with LLMs: A Study in Citation Text Generation](https://mlsanity.com/paper/2407.04046/) (2024)
- [An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's Guide](https://mlsanity.com/paper/2402.14837/) (2024)
- [Role-Play Zero-Shot Prompting with Large Language Models for Open-Domain Human-Machine Conversation](https://mlsanity.com/paper/2406.18460/) (2024)
- [CAPO: Cost-Aware Prompt Optimization](https://mlsanity.com/paper/2504.16005/) (2025)
- [Should LLM Safety Be More Than Refusing Harmful Instructions?](https://mlsanity.com/paper/2506.02442/) (2025)
- [Instruct or Interact? Exploring and Eliciting LLMs' Capability in Code Snippet Adaptation Through Prompt Engineering](https://mlsanity.com/paper/2411.15501/) (2024)
- [promptolution: A Unified, Modular Framework for Prompt Optimization](https://mlsanity.com/paper/2512.02840/) (2025)
- [Benchmarking Large Language Models for Cryptanalysis and Side-Channel Vulnerabilities](https://mlsanity.com/paper/2505.24621/) (2025)
- [Automating Knowledge Acquisition for Content-Centric Cognitive Agents Using LLMs](https://mlsanity.com/paper/2312.16378/) (2023)
- [Analyzing Large Language Models for Classroom Discussion Assessment](https://mlsanity.com/paper/2406.08680/) (2024)
- [From Extraction to Synthesis: Entangled Heuristics for Agent-Augmented Strategic Reasoning](https://mlsanity.com/paper/2507.13768/) (2025)
- [FaNS: a Facet-based Narrative Similarity Metric](https://mlsanity.com/paper/2309.04823/) (2023)
- [Assessing LLMs Suitability for Knowledge Graph Completion](https://mlsanity.com/paper/2405.17249/) (2024)
- [Temporal Validity Change Prediction](https://mlsanity.com/paper/2401.00779/) (2024)
- [Is ChatGPT the Future of Causal Text Mining? A Comprehensive Evaluation and Analysis](https://mlsanity.com/paper/2402.14484/) (2024)
- [Benchmarking LLMs on the Semantic Overlap Summarization Task](https://mlsanity.com/paper/2402.17008/) (2024)
- [Generative transformations and patterns in LLM-native approaches for software verification and falsification](https://mlsanity.com/paper/2404.09384/) (2024)
- [PromptPrism: A Linguistically-Inspired Taxonomy for Prompts](https://mlsanity.com/paper/2505.12592/) (2025)
- [Large Language Model-enhanced Reinforcement Learning for Low-Altitude Economy Networking](https://mlsanity.com/paper/2505.21045/) (2025)

## Abstract (arXiv)

While LLMs have shown great success in understanding and generating text in traditional conversational settings, their potential for performing ill-defined complex tasks is largely under-studied. Indeed, we are yet to conduct comprehensive benchmarking studies with multiple LLMs that are exclusively focused on a complex task. However, conducting such benchmarking studies is challenging because of the large variations in LLMs' performance when different prompt types/styles are used and different degrees of detail are provided in the prompts. To address this issue, the paper proposes a general taxonomy that can be used to design prompts with specific properties in order to perform a wide range of complex tasks. This taxonomy will allow future benchmarking studies to report the specific categories of prompts used as part of the study, enabling meaningful comparisons across different studies. Also, by establishing a common standard through this taxonomy, researchers will be able to draw more accurate conclusions about LLMs' performance on a specific complex task.