mlsanityData to 7 Oct 2026

What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering

arXiv 2406.12334 ↗First posted 18 Jun 2024cs.LG cs.SEcited by 88

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) changed the way we design and interact with software systems. Their ability to process and extract information from text has drastically improved productivity in a number of routine tasks. Developers that want to include these models in their software stack, however, face a dreadful challenge: debugging LLMs' inconsistent behavior across minor variations of the prompt. We therefore introduce two metrics for classification tasks, namely sensitivity and consistency, which are complementary to task performance. First, sensitivity measures changes of predictions across rephrasings of the prompt, and does not require access to ground truth labels. Instead, consistency measures how predictions vary across rephrasings for elements of the same class. We perform an empirical comparison of these metrics on text classification tasks, using them as guideline for understanding failure modes of the LLM. Our hope is that sensitivity and consistency will be helpful to guide prompt engineering and obtain LLMs that balance robustness with performance.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). Builds on 26 of its 41 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

26 of 41 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  8. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  9. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  10. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  11. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  12. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 326
14 more
  1. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  2. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
  3. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
  4. 2022GitHub Copilot AI pair programmer: Asset or Liability?cited by 150
  5. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  6. 2024To Believe or Not to Believe Your LLMcited by 120
  7. 2023LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarkscited by 94
  8. 2024"I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trustcited by 67
  9. 2024Uncertainty in Language Models: Assessment through Rank-Calibrationcited by 40
  10. 2024Are Large Language Models Good Prompt Optimizers?cited by 37
  11. 2024Examining the robustness of LLM evaluation to the distributional assumptions of benchmarkscited by 31
  12. 2024AgentQuest: A Modular Benchmark Framework to Measure Progress and Improve LLM Agentscited by 27
  13. 2024The Entropy Enigma: Success and Failure of Entropy Minimizationcited by 24
  14. 2024Just rephrase it! Uncertainty estimation in closed-source language models via multiple rephrased queriescited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 43; 2026: 36.

88 papers.

Sort:

By area:

  1. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 96
  2. 2024Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form QAcited by 25
  3. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 23
  4. 2025The Rogue Scalpel: Activation Steering Compromises LLM Safetycited by 18
  5. 2025Talk Structurally, Act Hierarchically: A Collaborative Framework for LLM Multi-Agent Systemscited by 17
  6. 2025GraphEval: A Lightweight Graph-Based LLM Framework for Idea Evaluationcited by 17
  7. 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systemscited by 16
  8. 2024Evaluating Gender Bias of LLMs in Making Morality Judgementscited by 15
  9. 2025Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Stylescited by 15
  10. 2025Prompt-Reverse Inconsistency: LLM Self-Inconsistency Beyond Generative Randomness and Prompt Paraphrasingcited by 14
  11. 2025Reinforcement Learning for Machine Learning Engineering Agentscited by 13
  12. 2025Promptception: How Sensitive Are Large Multimodal Models to Prompts?cited by 13
38 more
  1. 2025AutoSCORE: Enhancing Automated Scoring with Multi-Agent Large Language Models via Structured Component Recognitioncited by 13
  2. 2025Firm or Fickle? Evaluating Large Language Models Consistency in Sequential Interactionscited by 12
  3. 2024LLMs: A Game-Changer for Software Engineers?cited by 11
  4. 2025Distributive Fairness in Large Language Models: Evaluating Alignment with Human Valuescited by 9
  5. 2025PRL: Prompts from Reinforcement Learningcited by 9
  6. 2024Linguistic Features Extracted by GPT-4 Improve Alzheimer's Disease Detection based on Spontaneous Speechcited by 8
  7. 2025Meaningless Tokens, Meaningful Gains: How Activation Shifts Enhance LLM Reasoningcited by 8
  8. 2026The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?cited by 8
  9. 2026The Fragility Of Moral Judgment In Large Language Modelscited by 8
  10. 2025Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probingcited by 7
  11. 2024Privacy Bias in Language Models: A Contextual Integrity-based Auditing Metriccited by 6
  12. 2026ErrorMap and ErrorAtlas: Charting the Failure Landscape of Large Language Modelscited by 6
  13. 2024Leveraging Prior Experience: An Expandable Auxiliary Knowledge Base for Text-to-SQLcited by 5
  14. 2026Compositional Steering of Large Language Models with Steering Tokenscited by 5
  15. 2025AIRepr: An Analyst-Inspector Framework for Evaluating Reproducibility of LLMs in Data Sciencecited by 4
  16. 2025Cost-aware LLM-based Online Dataset Annotationcited by 4
  17. 2025Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG Evaluation Promptscited by 3
  18. 2025Natural Language Tools: A Natural Language Approach to Tool Calling In Large Language Agentscited by 3
  19. 2025GPS: General Per-Sample Promptercited by 3
  20. 2026Revisiting Prompt Sensitivity in Large Language Models for Text Classification: The Role of Prompt Underspecificationcited by 3
  21. 2025Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detectioncited by 2
  22. 2025LLMs as Data Annotators: How Close Are We to Human Performancecited by 2
  23. 2025Agents Require Metacognitive and Strategic Reasoning to Succeed in the Coming Labor Marketscited by 2
  24. 2025Question-to-Knowledge (Q2K): Multi-Agent Generation of Inspectable Facts for Product Mappingcited by 2
  25. 2025Is In-Context Learning Learning?cited by 2
  26. 2025PIAST: Rapid Prompting with In-context Augmentation for Scarce Training datacited by 2
  27. 2026How Open Must Language Models be to Enable Reliable Scientific Inference?cited by 2
  28. 2026Matryoshka Agent: Unfolding Sub-Agents for Long-Horizon Machine Learning Engineeringcited by 2
  29. 2024Talking with Tables for Better LLM Factual Data Interactionscited by 1
  30. 2025Understanding and supporting how developers prompt for LLM-powered code editing in practicecited by 1
  31. 2025CEA-LIST at CheckThat! 2025: Evaluating LLMs as Detectors of Bias and Opinion in Textcited by 1
  32. 2025Contrast Sensitivity in Multimodal Large Language Models: A Psychophysics-Inspired Evaluationcited by 1
  33. 2025Instance-level Randomization: Toward More Stable LLM Evaluationscited by 1
  34. 2025Position: AI Evaluations Should be Grounded on a Theory of Capabilitycited by 1
  35. 2025RCScore: Quantifying Response Consistency in Large Language Modelscited by 1
  36. 2025Prompt Fairness: Sub-group Disparities in LLMscited by 1
  37. 2025Focused Chain-of-Thought: Efficient LLM Reasoning via Structured Input Informationcited by 1
  38. 2025Advancing Text Classification with Large Language Models and Neural Attention Mechanismscited by 1

50 most cited of 88 All 88 (JSON)