mlsanityData to 7 Oct 2026

Why is prompting hard? Understanding prompts on binary sequence predictors

arXiv 2502.10760 ↗First posted 15 Feb 2025cs.CL cs.LG stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

Frontier models can be prompted or conditioned to do many tasks, but finding good prompts is not always easy, nor is understanding some performant prompts. We view prompting as finding the best conditioning sequence on a near-optimal sequence predictor. On numerous well-controlled experiments, we show that unintuitive optimal conditioning sequences can be better understood given the pretraining distribution, which is not usually available. Even using exhaustive search, reliably identifying optimal prompts for practical neural predictors can be surprisingly difficult. Popular prompting methods, such as using demonstrations from the targeted task, can be surprisingly suboptimal. Using the same empirical framework, we analyze optimal prompts on frontier models, revealing patterns similar to the binary examples and previous findings. Taken together, this work takes an initial step towards understanding optimal prompts, from a statistical and empirical perspective that complements research on frontier models.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 79 of its 134 references that are in the map. First posted on arXiv 15 Feb 2025.

Builds on

79 of 134 references.

References · 79

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  9. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  10. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  11. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  12. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
38 more
  1. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  2. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  3. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  4. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  5. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  6. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
  7. 2018Efficient Attention: Attention with Linear Complexitiescited by 366
  8. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  9. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  10. 2024xLSTM: Extended Long Short-Term Memorycited by 320
  11. 2023In-Context Learning Creates Task Vectorscited by 302
  12. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
  13. 2023Function Vectors in Large Language Modelscited by 292
  14. 2023How is ChatGPT's behavior changing over time?cited by 265
  15. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 264
  16. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  17. 2022Neural Networks and the Chomsky Hierarchycited by 214
  18. 2022Exploring Length Generalization in Large Language Modelscited by 204
  19. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  20. 2024Many-Shot In-Context Learningcited by 203
  21. 2023Large Language Models as General Pattern Machinescited by 196
  22. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 186
  23. 2023RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Modelscited by 173
  24. 2023ExpertPrompting: Instructing Large Language Models to be Distinguished Expertscited by 140
  25. 2024Does Prompt Formatting Have Any Impact on LLM Performance?cited by 132
  26. 2023The Learnability of In-Context Learningcited by 126
  27. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 95
  28. 2023How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?cited by 93
  29. 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agentscited by 89
  30. 2021Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuningcited by 82
  31. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 69
  32. 2019Meta-learning of Sequential Strategiescited by 68
  33. 2023Jailbreaking GPT-4V via Self-Adversarial Attacks with System Promptscited by 62
  34. 2024The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Modelscited by 62
  35. 2021Frequency Effects on Syntactic Rule Learning in Transformerscited by 59
  36. 2023How Many Demonstrations Do You Need for In-context Learning?cited by 52
  37. 2023The language of prompting: What linguistic properties make a prompt successful?cited by 43
  38. 2022Discovering the Hidden Vocabulary of DALLE-2cited by 42

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Understanding Prompt Tuning and In-Context Learning via Meta-Learningcited by 6
  2. 2025`A la recherche du sens perdu: your favourite LLM might have more to say than you can understandcited by 0
  3. 2026When Do We Need LLMs? A Diagnostic for Language-Driven Banditscited by 0
  4. 2026Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posteriorcited by 0