mlsanityData to 14 Oct 2026

TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Tasks

arXiv 2305.11430 ↗First posted 19 May 2023cs.AI cs.CL cs.IRcited by 49

Not on the map. Why?

Abstract (arXiv)

While LLMs have shown great success in understanding and generating text in traditional conversational settings, their potential for performing ill-defined complex tasks is largely under-studied. Indeed, we are yet to conduct comprehensive benchmarking studies with multiple LLMs that are exclusively focused on a complex task. However, conducting such benchmarking studies is challenging because of the large variations in LLMs' performance when different prompt types/styles are used and different degrees of detail are provided in the prompts. To address this issue, the paper proposes a general taxonomy that can be used to design prompts with specific properties in order to perform a wide range of complex tasks. This taxonomy will allow future benchmarking studies to report the specific categories of prompts used as part of the study, enabling meaningful comparisons across different studies. Also, by establishing a common standard through this taxonomy, researchers will be able to draw more accurate conclusions about LLMs' performance on a specific complex task.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Builds on 26 of its 43 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

26 of 43 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  10. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  11. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  12. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
14 more
  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  2. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  3. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  4. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 619
  5. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  6. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  7. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 388
  8. 2023Language Models can Solve Computer Taskscited by 372
  9. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  10. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  11. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  12. 2022Text and Patterns: For Effective Chain of Thought, It Takes Two to Tangocited by 128
  13. 2021MReD: A Meta-Review Dataset for Structure-Controllable Text Generationcited by 42
  14. 2023A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarizationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 20; 2025: 18; 2026: 6.

49 papers.

Sort:

By area:

  1. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 88
  2. 2024Determinants of LLM-assisted Decision-Makingcited by 56
  3. 2024UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them Allcited by 49
  4. 2024Argumentative Large Language Models for Explainable and Contestable Claim Verificationcited by 36
  5. 2023Is GPT-4 a reliable rater? Evaluating Consistency in GPT-4 Text Ratingscited by 34
  6. 2023Balancing Autonomy and Alignment: A Multi-Dimensional Taxonomy for Autonomous LLM-powered Multi-Agent Architecturescited by 25
  7. 2024iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvementcited by 23
  8. 2023ChatGPT as your Personal Data Scientistcited by 22
  9. 2024Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applicationscited by 17
  10. 2024LLMs as Meta-Reviewers' Assistants: A Case Studycited by 16
  11. 2025Taxonomy of User Needs and Actionscited by 15
  12. 2024Systematic Task Exploration with LLMs: A Study in Citation Text Generationcited by 8
37 more
  1. 2024An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's Guidecited by 7
  2. 2024Role-Play Zero-Shot Prompting with Large Language Models for Open-Domain Human-Machine Conversationcited by 7
  3. 2025CAPO: Cost-Aware Prompt Optimizationcited by 7
  4. 2025Should LLM Safety Be More Than Refusing Harmful Instructions?cited by 7
  5. 2024Instruct or Interact? Exploring and Eliciting LLMs' Capability in Code Snippet Adaptation Through Prompt Engineeringcited by 6
  6. 2025promptolution: A Unified, Modular Framework for Prompt Optimizationcited by 6
  7. 2025Benchmarking Large Language Models for Cryptanalysis and Side-Channel Vulnerabilitiescited by 5
  8. 2023Automating Knowledge Acquisition for Content-Centric Cognitive Agents Using LLMscited by 4
  9. 2024Analyzing Large Language Models for Classroom Discussion Assessmentcited by 4
  10. 2025From Extraction to Synthesis: Entangled Heuristics for Agent-Augmented Strategic Reasoningcited by 4
  11. 2023FaNS: a Facet-based Narrative Similarity Metriccited by 3
  12. 2024Assessing LLMs Suitability for Knowledge Graph Completioncited by 3
  13. 2024Temporal Validity Change Predictioncited by 2
  14. 2024Is ChatGPT the Future of Causal Text Mining? A Comprehensive Evaluation and Analysiscited by 2
  15. 2024Benchmarking LLMs on the Semantic Overlap Summarization Taskcited by 2
  16. 2024Generative transformations and patterns in LLM-native approaches for software verification and falsificationcited by 2
  17. 2025PromptPrism: A Linguistically-Inspired Taxonomy for Promptscited by 2
  18. 2025Large Language Model-enhanced Reinforcement Learning for Low-Altitude Economy Networkingcited by 2
  19. 2026OmniToM: Benchmarking Theory of Mind in LLMs via Explicit Belief Modelingcited by 2
  20. 2024Gradable ChatGPT Translation Evaluationcited by 1
  21. 2024WisPerMed at BioLaySumm: Adapting Autoregressive Large Language Models for Lay Summarization of Scientific Articlescited by 1
  22. 2024Extracting chemical food safety hazards from the scientific literature automatically using large language modelscited by 1
  23. 2025LLM for Complex Reasoning Task: An Exploratory Study in Fermi Problemscited by 1
  24. 2025LLM for Comparative Narrative Analysiscited by 1
  25. 2025Large Language Models for IT Automation Tasks: Are We There Yet?cited by 1
  26. 2026Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Taskscited by 1
  27. 2025To Google or To ChatGPT? A Comparison of CS2 Students' Information Gathering Approaches and Outcomescited by 0
  28. 2025Prompting a Weighting Mechanism into LLM-as-a-Judge in Two-Step: A Case Studycited by 0
  29. 2025Systems-Theoretic and Data-Driven Security Analysis in ML-enabled Medical Devicescited by 0
  30. 2025LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documentscited by 0
  31. 2025Beyond Jailbreak: Unveiling Risks in LLM Applications Arising from Blurred Capability Boundariescited by 0
  32. 2025DiscoVerse: Multi-Agent Pharmaceutical Co-Scientist for Traceable Drug Discovery and Reverse Translationcited by 0
  33. 2025DL3M: A Vision-to-Language Framework for Expert-Level Medical Reasoning through Deep Learning and Large Language Modelscited by 0
  34. 2026Are LLMs Ready to Replace Bangla Annotators?cited by 0
  35. 2026Exploring LLMs for South Asian Music Understanding and Generationcited by 0
  36. 2026Automatically Evolving Prompt Guidelines for Task-Specific Optimizationcited by 0
  37. 2026Evaluating Whether LLMs Can Reliably Connect the DOTs?cited by 0