mlsanityData to 14 Oct 2026

Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problem

arXiv 2403.03558 ↗First posted 6 Mar 2024cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are highly effective in various natural language processing (NLP) tasks. However, they are susceptible to producing unreliable conjectures in ambiguous contexts called hallucination. This paper presents a new method for evaluating LLM hallucination in Question Answering (QA) based on the unanswerable math word problem (MWP). To support this approach, we innovatively develop a dataset called Unanswerable Math Word Problem (UMWP) which comprises 5200 questions across five categories. We developed an evaluation methodology combining text similarity and mathematical expression detection to determine whether LLM considers the question unanswerable. The results of extensive experiments conducted on 31 LLMs, including GPT-3, InstructGPT, LLaMA, and Claude, demonstrate that in-context learning and reinforcement learning with human feedback (RLHF) training significantly enhance the model's ability to avoid hallucination. We show that utilizing MWP is a reliable and effective approach to assess hallucination. Our code and data are available at https://github.com/Yuki-Asuuna/UMWP.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Builds on 15 of its 23 references that are in the map. First posted on arXiv 6 Mar 2024.

Builds on

15 of 23 references.

References · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  9. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  10. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  11. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  12. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
3 more
  1. 2023A Survey of Hallucination in Large Foundation Modelscited by 368
  2. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  3. 2023Do Large Language Models Know What They Don't Know?cited by 243

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 21; 2026: 14.

43 papers.

Sort:

By area:

  1. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 244
  2. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
  3. 2025HalluLens: LLM Hallucination Benchmarkcited by 92
  4. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 65
  5. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 51
  6. 2024Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklistcited by 48
  7. 2025The Hallucination Tax of Reinforcement Finetuningcited by 40
  8. 2025Benchmarking Reasoning Robustness in Large Language Modelscited by 30
  9. 2025BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMscited by 27
  10. 2025MetaFaith: Faithful Natural Language Uncertainty Expression in LLMscited by 16
  11. 2026Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RLcited by 14
  12. 2025TreeCut: A Synthetic Unanswerable Math Word Problem Dataset for LLM Hallucination Evaluationcited by 12
31 more
  1. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  2. 2024When Context Leads but Parametric Memory Follows in Large Language Modelscited by 11
  3. 2024CheckEmbed: Effective Verification of LLM Solutions to Open-Ended Taskscited by 10
  4. 2024A Survey of Automatic Hallucination Evaluation on Natural Language Generationcited by 7
  5. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 7
  6. 2025Neural Message-Passing on Attention Graphs for Hallucination Detectioncited by 6
  7. 2026Statistical Early Stopping for Reasoning Modelscited by 6
  8. 2025Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directionscited by 5
  9. 2025Beyond Passive Critical Thinking: Fostering Proactive Questioning to Enhance Human-AI Collaborationcited by 5
  10. 2025Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitationcited by 5
  11. 2026Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMscited by 4
  12. 2025Investigating Symbolic Triggers of Hallucination in Gemma Models Across HaluEval and TruthfulQAcited by 3
  13. 2024When Not to Answer: Evaluating Prompts on GPT Models for Effective Abstention in Unanswerable Math Word Problemscited by 2
  14. 2024A Debate-Driven Experiment on LLM Hallucinations and Accuracycited by 2
  15. 2025LLMEval-Fair: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Modelscited by 2
  16. 2025Investigating Retrieval-Augmented Generation Systems on Unanswerable, Uncheatable, Realistic, Multi-hop Queriescited by 2
  17. 2025Language Models Learn Universal Representations of Numbers and Here's Why You Should Carecited by 2
  18. 2026CARE-RFT: Confidence-Anchored Reinforcement Finetuning for Reliable Reasoning in Large Language Modelscited by 2
  19. 2026Beyond "I Don't Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertaintycited by 2
  20. 2025VisionTrap: Unanswerable Questions On Visual Datacited by 1
  21. 2025HalluClean: A Unified Framework to Combat Hallucinations in LLMscited by 1
  22. 2026Talking to a Know-It-All GPT or a Second-Guesser Claude? How Repair reveals unreliable Multi-Turn Behavior in LLMscited by 1
  23. 2026Enhancing LLM Metacognition via Cognitive Pairwise Trainingcited by 1
  24. 2025Machine Mirages: Defining the Undefinedcited by 0
  25. 2026Bridging the Know-Act Gap via Task-Level Autoregressive Reasoningcited by 0
  26. 2026Attention Sinks as Internal Signals for Hallucination Detection in Large Language Modelscited by 0
  27. 2026HalluScan: A Systematic Benchmark for Detecting and Mitigating Hallucinations in Instruction-Following LLMscited by 0
  28. 2026Ask, Condition or Abstain: Reinforcement Learning for Missing-Premise Reasoningcited by 0
  29. 2026More Capable, Less Faithful: A Multilingual Analysis of Mathematical (Un)Solvability Detection in LLMscited by 0
  30. 2026Solving versus Verifying: Catching Contradictions in Tax Reasoning Systemscited by 0
  31. 2026Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Hallucination-Free Automatic Question & Answer Generation for Intuitive Learning
  2. 2023ReEval: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attacks
  3. 2025Enhancing Reliability across Short and Long-Form QA via Reinforcement Learning
  4. 2024Investigating and Addressing Hallucinations of LLMs in Tasks Involving Negation
  5. 2025SelfCheck-Eval: A Multi-Module Framework for Zero-Resource Hallucination Detection in Large Language Models
  6. 2026Evaluating Hallucinations in Domain-Adapted Large Language Models
  7. 2025Learned Hallucination Detection in Black-Box LLMs using Token-level Entropy Production Rate
  8. 2024HaluEval-Wild: Evaluating Hallucinations of Language Models in the Wild