# Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absent

arXiv: https://arxiv.org/abs/2502.11267 · HTML: https://mlsanity.com/paper/2502.11267/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 121 references that are in the map.
- First posted on arXiv 16 Feb 2025.

## Builds on (most cited first, 30 of 45)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024)
- [AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators](https://mlsanity.com/paper/2303.16854/) (2023)
- [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](https://mlsanity.com/paper/2102.09692/) (2021)
- [Explanations Can Reduce Overreliance on AI Systems During Decision-Making](https://mlsanity.com/paper/2212.06823/) (2022)
- [Improving Text-to-Image Consistency via Automatic Prompt Optimization](https://mlsanity.com/paper/2403.17804/) (2024)
- [ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing](https://mlsanity.com/paper/2309.09128/) (2023)
- [LLMaAA: Making Large Language Models as Active Annotators](https://mlsanity.com/paper/2310.19596/) (2023)
- [Review of Large Vision Models and Visual Prompt Engineering](https://mlsanity.com/paper/2307.00855/) (2023)
- [LLMs Accelerate Annotation for Medical Information Extraction](https://mlsanity.com/paper/2312.02296/) (2023)
- [Language Model Self-improvement by Reinforcement Learning Contemplation](https://mlsanity.com/paper/2305.14483/) (2023)
- [LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis](https://mlsanity.com/paper/2310.15100/) (2023)
- [Beyond Fair Pay: Ethical Implications of NLP Crowdsourcing](https://mlsanity.com/paper/2104.10097/) (2021)
- [Leveraging Explanations in Interactive Machine Learning: An Overview](https://mlsanity.com/paper/2207.14526/) (2022)
- [MEGAnno+: A Human-LLM Collaborative Annotation System](https://mlsanity.com/paper/2402.18050/) (2024)
- [How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labels](https://mlsanity.com/paper/2008.11721/) (2020)
- [If in a Crowdsourced Data Annotation Pipeline, a GPT-4](https://mlsanity.com/paper/2402.16795/) (2024)
- [CollabCoder: A Lower-barrier, Rigorous Workflow for Inductive Collaborative Qualitative Analysis with Large Language Models](https://mlsanity.com/paper/2304.07366/) (2023)
- [AutoHint: Automatic Prompt Optimization with Hint Generation](https://mlsanity.com/paper/2307.07415/) (2023)
- [Evaluating Explanation Without Ground Truth in Interpretable Machine Learning](https://mlsanity.com/paper/1907.06831/) (2019)
- [Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimization](https://mlsanity.com/paper/2406.15708/) (2024)
- [A Comparative Study on Annotation Quality of Crowdsourcing and LLM via Label Aggregation](https://mlsanity.com/paper/2401.09760/) (2024)
- [Modeling and mitigating human annotation errors to design efficient stream processing systems with human-in-the-loop machine learning](https://mlsanity.com/paper/2007.03177/) (2020)
- [Human-In-The-Loop Person Re-Identification](https://mlsanity.com/paper/1612.01345/) (2016)
- [The Promises and Pitfalls of LLM Annotations in Dataset Labeling: a Case Study on Media Bias Detection](https://mlsanity.com/paper/2411.11081/) (2024)
- [Intent-based Prompt Calibration: Enhancing prompt optimization with synthetic boundary cases](https://mlsanity.com/paper/2402.03099/) (2024)

## Built on by (most cited first, 15 of 15)

- [Show or Tell? Modeling the evolution of request-making in Human-LLM conversations](https://mlsanity.com/paper/2508.01213/) (2025)
- [How well do Large Language Models Recognize Instructional Moves? Establishing Baselines for Foundation Models in Educational Discourse](https://mlsanity.com/paper/2512.19903/) (2025)
- [The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation](https://mlsanity.com/paper/2602.11318/) (2026)
- [Measurement as Bricolage: Examining How Data Scientists Construct Target Variables for Predictive Modeling Tasks](https://mlsanity.com/paper/2507.02819/) (2025)
- [Prompt-with-Me: in-IDE Structured Prompt Management for LLM-Driven Software Engineering](https://mlsanity.com/paper/2509.17096/) (2025)
- [Data-Prompt Co-Evolution: Growing Test Sets to Refine LLM Behavior](https://mlsanity.com/paper/2510.12728/) (2025)
- [Sandpiper: Orchestrated AI-Annotation for Educational Discourse at Scale](https://mlsanity.com/paper/2603.08406/) (2026)
- [Priming, Path-dependence, and Plasticity: Understanding the molding of user-LLM interaction and its implications from (many) chat logs in the wild](https://mlsanity.com/paper/2605.05767/) (2026)
- [JMMMU-Pro: Image-based Japanese Multi-discipline Multimodal Understanding Benchmark via Vibe Benchmark Construction](https://mlsanity.com/paper/2512.14620/) (2025)
- [PromptHelper: A Prompt Recommender System for Encouraging Creativity in AI Chatbot Interactions](https://mlsanity.com/paper/2601.15575/) (2026)
- [To Write or to Automate Linguistic Prompts, That Is the Question](https://mlsanity.com/paper/2603.25169/) (2026)
- [Beyond One Output: Visualizing and Comparing Distributions of Language Model Generations](https://mlsanity.com/paper/2604.18724/) (2026)
- [Locating acts of mechanistic reasoning in student team conversations with mechanistic machine learning](https://mlsanity.com/paper/2604.21870/) (2026)
- [Experts Rise Where LLMs Disagree: Using Cross-Model Disagreement to Target Expert Effort in LLM Codebook Revision for Large-Scale Annotation](https://mlsanity.com/paper/2609.26926/) (2026)
- [EduBehaviors: Assertion-based Schemas for Auditable Coding of Educational Dialogues](https://mlsanity.com/paper/2609.27043/) (2026)

## Abstract (arXiv)

Millions of users prompt large language models (LLMs) for various tasks, but how good are people at prompt engineering? Do users actually get closer to their desired outcome over multiple iterations of their prompts? These questions are crucial when no gold-standard labels are available to measure progress. This paper investigates a scenario in LLM-powered data labeling, "prompting in the dark," where users iteratively prompt LLMs to label data without using manually-labeled benchmarks. We developed PromptingSheet, a Google Sheets add-on that enables users to compose, revise, and iteratively label data through spreadsheets. Through a study with 20 participants, we found that prompting in the dark was highly unreliable -- only 9 participants improved labeling accuracy after four or more iterations. Automated prompt optimization tools like DSPy also struggled when few gold labels were available. Our findings highlight the importance of gold labels and the needs, as well as the risks, of automated support in human prompt engineering, providing insights for future tool design.