mlsanityData to 14 Oct 2026

ClozeMath: Improving Mathematical Reasoning in Language Models by Learning to Fill Equations

arXiv 2506.03763 ↗First posted 4 Jun 2025cs.CLcited by 2

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

The capabilities of large language models (LLMs) have been enhanced by training on data that reflects human thought processes, such as the Chain-of-Thought format. However, evidence suggests that the conventional scheme of next-word prediction may not fully capture how humans learn to think. Inspired by how humans generalize mathematical reasoning, we propose a new approach named ClozeMath to fine-tune LLMs for mathematical reasoning. Our ClozeMath involves a text-infilling task that predicts masked equations from a given solution, analogous to cloze exercises used in human learning. Experiments on GSM8K, MATH, and GSM-Symbolic show that ClozeMath surpasses the strong baseline Masked Thought in performance and robustness, with two test-time scaling decoding algorithms, Beam Search and Chain-of-Thought decoding. Additionally, we conduct an ablation study to analyze the effects of various architectural and implementation choices on our approach.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 3 of its 24 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

3 of 24 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models5 Feb

Other references · 2

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Masked-and-Reordered Self-Supervision for Reinforcement Learning from Verifiable Rewardscited by 5
  2. 2026When Tokenization is Secretly Output Supervisioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Large Language Models for Mathematical Reasoning
  2. 2024MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time
  3. 2025Thinking Machines: Mathematical Reasoning in the Age of LLMs
  4. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  5. 2024Assessing the Emergent Symbolic Reasoning Abilities of Llama Large Language Models
  6. 2024Large Language Models for Mathematical Reasoning: Progresses and Challenges
  7. 2024Benchmarking Large Language Models for Math Reasoning Tasks
  8. 2024ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline