mlsanityData to 14 Oct 2026

Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absent

arXiv 2502.11267 ↗First posted 16 Feb 2025cs.HC cs.AI cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Millions of users prompt large language models (LLMs) for various tasks, but how good are people at prompt engineering? Do users actually get closer to their desired outcome over multiple iterations of their prompts? These questions are crucial when no gold-standard labels are available to measure progress. This paper investigates a scenario in LLM-powered data labeling, "prompting in the dark," where users iteratively prompt LLMs to label data without using manually-labeled benchmarks. We developed PromptingSheet, a Google Sheets add-on that enables users to compose, revise, and iteratively label data through spreadsheets. Through a study with 20 participants, we found that prompting in the dark was highly unreliable -- only 9 participants improved labeling accuracy after four or more iterations. Automated prompt optimization tools like DSPy also struggled when few gold labels were available. Our findings highlight the importance of gold labels and the needs, as well as the risks, of automated support in human prompt engineering, providing insights for future tool design.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 45 of its 121 references that are in the map. First posted on arXiv 16 Feb 2025.

Builds on

45 of 121 references.

References · 45

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  3. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  4. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  5. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  6. 2024Large Language Models for Data Annotation and Synthesis: A Surveycited by 205
  7. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 172
  8. 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
  9. 2022Explanations Can Reduce Overreliance on AI Systems During Decision-Makingcited by 111
  10. 2024Improving Text-to-Image Consistency via Automatic Prompt Optimizationcited by 90
  11. 2023ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testingcited by 88
  12. 2023LLMaAA: Making Large Language Models as Active Annotatorscited by 79
33 more
  1. 2023Review of Large Vision Models and Visual Prompt Engineeringcited by 77
  2. 2023LLMs Accelerate Annotation for Medical Information Extractioncited by 77
  3. 2023Language Model Self-improvement by Reinforcement Learning Contemplationcited by 75
  4. 2023LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysiscited by 70
  5. 2021Beyond Fair Pay: Ethical Implications of NLP Crowdsourcingcited by 53
  6. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 53
  7. 2024MEGAnno+: A Human-LLM Collaborative Annotation Systemcited by 47
  8. 2020How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labelscited by 39
  9. 2024If in a Crowdsourced Data Annotation Pipeline, a GPT-4cited by 38
  10. 2023CollabCoder: A Lower-barrier, Rigorous Workflow for Inductive Collaborative Qualitative Analysis with Large Language Modelscited by 34
  11. 2023AutoHint: Automatic Prompt Optimization with Hint Generationcited by 30
  12. 2019Evaluating Explanation Without Ground Truth in Interpretable Machine Learningcited by 29
  13. 2024Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimizationcited by 29
  14. 2024A Comparative Study on Annotation Quality of Crowdsourcing and LLM via Label Aggregationcited by 23
  15. 2020Modeling and mitigating human annotation errors to design efficient stream processing systems with human-in-the-loop machine learningcited by 21
  16. 2016Human-In-The-Loop Person Re-Identificationcited by 19
  17. 2024The Promises and Pitfalls of LLM Annotations in Dataset Labeling: a Case Study on Media Bias Detectioncited by 15
  18. 2024Intent-based Prompt Calibration: Enhancing prompt optimization with synthetic boundary casescited by 13
  19. 2021A Privacy-Preserving Approach to Extraction of Personal Information through Automatic Annotation and Federated Learningcited by 12
  20. 2021A Survey on Task Assignment in Crowdsourcingcited by 8
  21. 2024Clarify: Improving Model Robustness With Natural Language Correctionscited by 8
  22. 2024End User Authoring of Personalized Content Classifiers: Comparing Example Labeling, Rule Writing, and LLM Promptingcited by 8
  23. 2024CoPrompter: User-Centric Evaluation of LLM Instruction Alignment for Improved Prompt Engineeringcited by 8
  24. 2020Learnability with Indirect Supervision Signalscited by 7
  25. 2023Revisiting Automated Prompting: Are We Actually Doing Better?cited by 7
  26. 2024GRAD-SUM: Leveraging Gradient Summarization for Optimal Prompt Engineeringcited by 7
  27. 2021The Challenge of Variable Effort Crowdsourcing and How Visible Gold Can Helpcited by 6
  28. 2023Capture the Flag: Uncovering Data Insights with Large Language Modelscited by 4
  29. 2024Harnessing LLMs for Automated Video Content Analysis: An Exploratory Workflow of Short Videos on Depressioncited by 4
  30. 2024Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labelscited by 4
  31. 2014How Many Workers to Ask? Adaptive Exploration for Collecting High Quality Labelscited by 3
  32. 2024GLaPE: Gold Label-agnostic Prompt Evaluation and Optimization for Large Language Modelcited by 3
  33. 2023Labeling without Seeing? Blind Annotation for Privacy-Preserving Entity Resolutioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 9.

15 papers.

Sort:

  1. 2025Show or Tell? Modeling the evolution of request-making in Human-LLM conversationscited by 3
  2. 2025How well do Large Language Models Recognize Instructional Moves? Establishing Baselines for Foundation Models in Educational Discoursecited by 2
  3. 2026The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotationcited by 2
  4. 2025Measurement as Bricolage: Examining How Data Scientists Construct Target Variables for Predictive Modeling Taskscited by 1
  5. 2025Prompt-with-Me: in-IDE Structured Prompt Management for LLM-Driven Software Engineeringcited by 1
  6. 2025Data-Prompt Co-Evolution: Growing Test Sets to Refine LLM Behaviorcited by 1
  7. 2026Sandpiper: Orchestrated AI-Annotation for Educational Discourse at Scalecited by 1
  8. 2026Priming, Path-dependence, and Plasticity: Understanding the molding of user-LLM interaction and its implications from (many) chat logs in the wildcited by 1
  9. 2025JMMMU-Pro: Image-based Japanese Multi-discipline Multimodal Understanding Benchmark via Vibe Benchmark Constructioncited by 0
  10. 2026PromptHelper: A Prompt Recommender System for Encouraging Creativity in AI Chatbot Interactionscited by 0
  11. 2026To Write or to Automate Linguistic Prompts, That Is the Questioncited by 0
  12. 2026Beyond One Output: Visualizing and Comparing Distributions of Language Model Generationscited by 0
3 more
  1. 2026Locating acts of mechanistic reasoning in student team conversations with mechanistic machine learningcited by 0
  2. 2026Experts Rise Where LLMs Disagree: Using Cross-Model Disagreement to Target Expert Effort in LLM Codebook Revision for Large-Scale Annotationcited by 0
  3. 2026EduBehaviors: Assertion-based Schemas for Auditable Coding of Educational Dialoguescited by 0