# Let's Verify Math Questions Step by Step

arXiv: https://arxiv.org/abs/2505.13903 · HTML: https://mlsanity.com/paper/2505.13903/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Mathematical reasoning, one of 17 directions in LLM reasoning.
- Builds on 4 of its 7 references that are in the map.
- First posted on arXiv 20 May 2025.

## Placed in

- LLM reasoning / [Mathematical reasoning](https://mlsanity.com/lr/mathematical-reasoning/)

## Builds on (most cited first, 4 of 4)

- [MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs](https://mlsanity.com/paper/2402.16352/) (2024)
- [Large language models surpass human experts in predicting neuroscience results](https://mlsanity.com/paper/2403.03230/) (2024)
- [On the Creativity of Large Language Models](https://mlsanity.com/paper/2304.00008/) (2023)
- [TreeCut: A Synthetic Unanswerable Math Word Problem Dataset for LLM Hallucination Evaluation](https://mlsanity.com/paper/2502.13442/) (2025)

## Built on by (most cited first, 18 of 18)

- [DataFlow: An LLM-Driven Framework for Unified Data Preparation and Workflow Automation in the Era of Data-Centric AI](https://mlsanity.com/paper/2512.16676/) (2025)
- [From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations](https://mlsanity.com/paper/2509.16584/) (2025)
- [GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Models](https://mlsanity.com/paper/2511.11134/) (2025)
- [AgenticRAGTracer: A Hop-Aware Benchmark for Diagnosing Multi-Step Retrieval Reasoning in Agentic RAG](https://mlsanity.com/paper/2602.19127/) (2026)
- [LongVidSearch: An Agentic Benchmark for Multi-hop Evidence Retrieval Planning in Long Videos](https://mlsanity.com/paper/2603.14468/) (2026)
- [Can Large Multimodal Models Actively Recognize Faulty Inputs? A Systematic Evaluation Framework of Their Input Scrutiny Ability](https://mlsanity.com/paper/2508.04017/) (2025)
- [Towards Next-Generation LLM Training: From the Data-Centric Perspective](https://mlsanity.com/paper/2603.14712/) (2026)
- [OpenWorldLib: A Unified Codebase and Definition of Advanced World Models](https://mlsanity.com/paper/2604.04707/) (2026)
- [Rethinking Math Reasoning Evaluation: A Robust LLM-as-a-Judge Framework Beyond Symbolic Rigidity](https://mlsanity.com/paper/2604.22597/) (2026)
- [GraphPO: Graph-based Policy Optimization for Reasoning Models](https://mlsanity.com/paper/2606.18954/) (2026)
- [Judge Before Answer: Can MLLM Discern the False Premise in Question?](https://mlsanity.com/paper/2510.10965/) (2025)
- [MorphoBench: A Benchmark with Difficulty Adaptive to Model Reasoning](https://mlsanity.com/paper/2510.14265/) (2025)
- [MathMixup: Boosting LLM Mathematical Reasoning with Difficulty-Controllable Data Synthesis and Curriculum Learning](https://mlsanity.com/paper/2601.17006/) (2026)
- [RREDCoT: Segment-Level Reward Redistribution for Reasoning Models](https://mlsanity.com/paper/2606.06475/) (2026)
- [DataPrep-Bench: Benchmarking LLMs as Training Data Preparators](https://mlsanity.com/paper/2607.20465/) (2026)
- [Hard Negative Sample-Augmented DPO Post-Training for Small Language Models](https://mlsanity.com/paper/2512.19728/) (2025)
- [Guided Verifier: Collaborative Multimodal Reasoning via Dynamic Process Supervision](https://mlsanity.com/paper/2602.04290/) (2026)
- [Questioning the Questions: Sustaining Self-Evolution in Reasoning Models](https://mlsanity.com/paper/2610.04299/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have recently achieved remarkable progress in mathematical reasoning. To enable such capabilities, many existing works distill strong reasoning models into long chains of thought or design algorithms to construct high-quality math question-answer (QA) data for training. However, these efforts primarily focus on generating correct reasoning paths and answers, while largely overlooking the correctness of the questions themselves. In this work, we present ValiMath, a benchmark consisting of 2147 human-verified mathematical questions covering a wide range of domains such as arithmetic, algebra, and geometry, which are synthesized and curated from the NuminaMath dataset. Each question is annotated with its logical structure, domain coverage, and question correctness, enabling fine-grained evaluation of question quality. ValiMath serves as a high-quality gold-standard test set for validating mathematical questions in LLM training corpora. Building upon this benchmark, we further propose MathQ-Verify, a pipeline that performs fine-grained parsing of mathematical questions into atomic assumptions and conclusions, and evaluates their semantic soundness through consistency checks. This pipeline achieves high precision in detecting flawed questions and provides a reliable foundation for cleaning noisy mathematical datasets. Experiments show that MathQ-Verify achieves state-of-the-art performance across multiple benchmarks, improving the F1 score by up to 25 percentage points over the direct verification baseline. MathQ-Verify offers a scalable and accurate solution for curating reliable mathematical datasets, reducing label noise and avoiding unnecessary computation on invalid questions. Our code and data are available at the repository https://github.com/OpenDCAI/MathQ-Verify.