mlsanityData to 14 Oct 2026

Prompting as Scientific Inquiry

arXiv 2507.00163 ↗First posted 30 Jun 2025cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Prompting is the primary method by which we study and control large language models. It is also one of the most powerful: nearly every major capability attributed to LLMs-few-shot learning, chain-of-thought, constitutional AI-was first unlocked through prompting. Yet prompting is rarely treated as science and is frequently frowned upon as alchemy. We argue that this is a category error. If we treat LLMs as a new kind of complex and opaque organism that is trained rather than programmed, then prompting is not a workaround: it is behavioral science. Mechanistic interpretability peers into the neural substrate, prompting probes the model in its native interface: language. We contend that prompting is not inferior, but rather a key component in the science of LLMs.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 43 of its 61 references that are in the map. First posted on arXiv 30 Jun 2025.

Builds on

43 of 61 references.

References · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  11. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  12. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
31 more
  1. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  2. 2022Editing Models with Task Arithmeticcited by 1,120
  3. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  4. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  5. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  6. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  7. 2024Scaling and evaluating sparse autoencoderscited by 553
  8. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  9. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  10. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 329
  11. 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 230
  12. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Modelscited by 191
  13. 2023Mitigating the Alignment Tax of RLHFcited by 183
  14. 2021The Values Encoded in Machine Learning Researchcited by 172
  15. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 156
  16. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  17. 2018Character-Level Language Modeling with Deeper Self-Attentioncited by 138
  18. 2022Language Models as Agent Modelscited by 132
  19. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  20. 2018Troubling Trends in Machine Learning Scholarshipcited by 108
  21. 2025Sparse Autoencoders Trained on the Same Data Learn Different Featurescited by 93
  22. 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 86
  23. 2025Language Models Use Trigonometry to Do Additioncited by 70
  24. 2021On the Diversity and Limits of Human Explanationscited by 30
  25. 2025We Can't Understand AI Using our Existing Vocabularycited by 25
  26. 2023Bayes in the age of intelligent machinescited by 22
  27. 2023Generative Models as a Complex Systems Science: How can we make sense of large language model behavior?cited by 19
  28. 2024From Prompt Engineering to Prompt Science With Human in the Loopcited by 12
  29. 2025Prompting Science Report 1: Prompt Engineering is Complicated and Contingentcited by 8
  30. 2025HyPerAlign: Interpretable Personalized LLM Alignment via Hypothesis Generationcited by 7
  31. 2025Prompt-Hacking: The New p-Hacking?cited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Is Your Agent Playing Dead? Deployed LLM Agents Exhibit Constraint-Evasive Fabrication and Thanatosiscited by 0
  2. 2026An Investigation of the NeurIPS and ICML 2025 Position Trackscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Agent-Centric Projection of Prompting Techniques and Implications for Synthetic Training Data for Large Language Models
  2. 2024Towards Generalist Prompting for Large Language Models by Mental Models
  3. 2026Beyond the Answer: Decoding the Behavior of LLMs as Scientific Reasoners
  4. 2026Understanding Large Language Models
  5. 2025You Don't Need Prompt Engineering Anymore: The Prompting Inversion
  6. 2024Rethinking ChatGPT's Success: Usability and Cognitive Behaviors Enabled by Auto-regressive LLMs' Prompting
  7. 2023Metacognitive Prompting Improves Understanding in Large Language Models
  8. 2025A Review of Developmental Interpretability in Large Language Models