# Neuro-Symbolic Data Generation for Math Reasoning

arXiv: https://arxiv.org/abs/2412.04857 · HTML: https://mlsanity.com/paper/2412.04857/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Mathematical reasoning, one of 17 directions in LLM reasoning.
- Builds on 15 of its 73 references that are in the map.
- First posted on arXiv 6 Dec 2024.

## Placed in

- LLM reasoning / [Mathematical reasoning](https://mlsanity.com/lr/mathematical-reasoning/)

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://mlsanity.com/paper/2309.12284/) (2023)
- [WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct](https://mlsanity.com/paper/2308.09583/) (2023)
- [Llemma: An Open Language Model For Mathematics](https://mlsanity.com/paper/2310.10631/) (2023)
- [ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving](https://mlsanity.com/paper/2309.17452/) (2023)
- [SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models](https://mlsanity.com/paper/2307.10635/) (2023)
- [AlphaMath Almost Zero: Process Supervision without Process](https://mlsanity.com/paper/2405.03553/) (2024)
- [MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning](https://mlsanity.com/paper/2310.03731/) (2023)
- [Forward-Backward Reasoning in Large Language Models for Mathematical Verification](https://mlsanity.com/paper/2308.07758/) (2023)
- [An Independent Evaluation of ChatGPT on Mathematical Word Problems (MWP)](https://mlsanity.com/paper/2302.13814/) (2023)
- [Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning](https://mlsanity.com/paper/2312.08901/) (2023)

## Built on by (most cited first, 22 of 22)

- [A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges](https://mlsanity.com/paper/2412.11936/) (2024)
- [Proving Olympiad Inequalities by Synergizing LLMs and Symbolic Reasoning](https://mlsanity.com/paper/2502.13834/) (2025)
- [A Survey on Mathematical Reasoning and Optimization with Large Language Models](https://mlsanity.com/paper/2503.17726/) (2025)
- [RE-IMAGINE: Symbolic Benchmark Synthesis for Reasoning Evaluation](https://mlsanity.com/paper/2506.15455/) (2025)
- [LawGPT: Knowledge-Guided Data Generation and Its Application to Legal LLM](https://mlsanity.com/paper/2502.06572/) (2025)
- [LLM-Powered Benchmark Factory: Reliable, Generic, and Efficient](https://mlsanity.com/paper/2502.01683/) (2025)
- [Unlocking the Potential of Generative AI through Neuro-Symbolic Architectures: Benefits and Limitations](https://mlsanity.com/paper/2502.11269/) (2025)
- [Mind the Generation Process: Fine-Grained Confidence Estimation During LLM Generation](https://mlsanity.com/paper/2508.12040/) (2025)
- [TabularMath: Understanding Math Reasoning over Tables with Large Language Models](https://mlsanity.com/paper/2505.19563/) (2025)
- [ASyMOB: Algebraic Symbolic Mathematical Operations Benchmark](https://mlsanity.com/paper/2505.23851/) (2025)
- [The Road to Generalizable Neuro-Symbolic Learning Should be Paved with Foundation Models](https://mlsanity.com/paper/2505.24874/) (2025)
- [LAST: Leveraging Tools as Hints to Enhance Spatial Reasoning for Multimodal Large Language Models](https://mlsanity.com/paper/2604.09712/) (2026)
- [Saturation-Driven Dataset Generation for LLM Mathematical Reasoning in the TPTP Ecosystem](https://mlsanity.com/paper/2509.06809/) (2025)
- [LexPro-1.0 Technical Report](https://mlsanity.com/paper/2503.06949/) (2025)
- [LoRA-PAR: A Flexible Dual-System LoRA Partitioning Approach to Efficient LLM Fine-Tuning](https://mlsanity.com/paper/2507.20999/) (2025)
- [From Static to Dynamic: Adaptive Monte Carlo Search for Mathematical Process Supervision](https://mlsanity.com/paper/2509.24351/) (2025)
- [Towards Camera-Robust 3D Localization: Equation-Anchored Tool-Use for MLLMs](https://mlsanity.com/paper/2605.19528/) (2026)
- [Symbolic Neural Generation with Applications to Lead Discovery in Drug Design](https://mlsanity.com/paper/2510.23379/) (2025)
- [Computational Blueprints: Generating Isomorphic Mathematics Problems with Large Language Models](https://mlsanity.com/paper/2511.07932/) (2025)
- [Adaptive Problem Generation via Symbolic Representations](https://mlsanity.com/paper/2602.19187/) (2026)
- [A Neuro-Symbolic Approach to Strategy Synthesis for Strategic Logics](https://mlsanity.com/paper/2606.17962/) (2026)
- [Agent Behavior as Code: Efficient and Robust LLM Agents with Programmatic Specifications](https://mlsanity.com/paper/2610.04824/) (2026)

## Abstract (arXiv)

A critical question about Large Language Models (LLMs) is whether their apparent deficiency in mathematical reasoning is inherent, or merely a result of insufficient exposure to high-quality mathematical data. To explore this, we developed an automated method for generating high-quality, supervised mathematical datasets. The method carefully mutates existing math problems, ensuring both diversity and validity of the newly generated problems. This is achieved by a neuro-symbolic data generation framework combining the intuitive informalization strengths of LLMs, and the precise symbolic reasoning of math solvers along with projected Markov chain Monte Carlo sampling in the highly-irregular symbolic space. Empirical experiments demonstrate the high quality of data generated by the proposed method, and that the LLMs, specifically LLaMA-2 and Mistral, when realigned with the generated data, surpass their state-of-the-art counterparts.