Ask-Before-Detection: Identifying and Mitigating Conformity Bias in LLM-Powered Error Detector for Math Word Problem Solutions
Not on the map. Why?
Abstract (arXiv)
The rise of large language models (LLMs) offers new opportunities for automatic error detection in education, particularly for math word problems (MWPs). While prior studies demonstrate the promise of LLMs as error detectors, they overlook the presence of multiple valid solutions for a single MWP. Our preliminary analysis reveals a significant performance gap between conventional and alternative solutions in MWPs, a phenomenon we term conformity bias in this work. To mitigate this bias, we introduce the Ask-Before-Detect (AskBD) framework, which generates adaptive reference solutions using LLMs to enhance error detection. Experiments on 200 examples of GSM8K show that AskBD effectively mitigates bias and improves performance, especially when combined with reasoning-enhancing techniques like chain-of-thought prompting.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 17 of its 32 references that are in the map. First posted on arXiv 22 Dec 2024.
Builds on
17 of 32 references.
References · 17
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2024Qwen2 Technical Reportcited by 2,076
- 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
- 2024Understanding the planning of LLM agents: A surveycited by 414
- 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
- 2023Using an LLM to Help With Code Understandingcited by 222
- 2024Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gapcited by 79
- 2024Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklistcited by 48
- 2024Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correctioncited by 45
5 more
- 2024Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutorscited by 39
- 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
- 2024LLMs can Find Mathematical Reasoning Mistakes by Pedagogical Chain-of-Thoughtcited by 28
- 2024Deciphering the Factors Influencing the Efficacy of Chain-of-Thought: Probability, Memorization, and Noisy Reasoningcited by 27
- 2023Three Questions Concerning the Use of Large Language Models to Facilitate Mathematics Learningcited by 9
Built on by
Citing papers per year: 2025: 3; 2026: 1.
4 papers.
- 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
- 2025MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detectioncited by 24
- 2026Maximizing Local Entropy Where It Matters: Prefix-Aware Localized LLM Unlearningcited by 10
- 2025LLMs cannot spot math errors, even when allowed to peek into the solutioncited by 7
- 2026Maximizing Local Entropy Where It Matters: Prefix-Aware Localized LLM Unlearning6 Jan 2026
- 2025LLMs cannot spot math errors, even when allowed to peek into the solution1 Sep 2025
- 2025MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection23 Mar 2025
- 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning5 Feb 2025
Similar, not cited
Close in wording, no citation either way
- 2025Error Classification of Large Language Models on Math Word Problems: A Dynamically Adaptive Framework
- 2024Exposing the Achilles' Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoning
- 2024Large Language Models Struggle with Unreasonability in Math Problems
- 2024Evaluating LLMs at Detecting Errors in LLM Responses
- 2025From Correctness to Comprehension: AI Agents for Personalized Error Diagnosis in Education
- 2025LEMMA: Learning from Errors for MatheMatical Advancement in LLMs
- 2026Improving Mathematical Reasoning Capabilities in Large Language Models via Reasoning Process Error Classification
- 2025SMRC: Aligning Large Language Models with Student Reasoning for Mathematical Error Correction