mlsanityData to 7 Oct 2026

LLMs for Mathematical Modeling: Towards Bridging the Gap between Natural and Mathematical Languages

arXiv 2405.13144 ↗First posted 21 May 2024cs.AI cs.CLcited by 46

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated strong performance across various natural language processing tasks, yet their proficiency in mathematical reasoning remains a key challenge. Addressing the gap between natural and mathematical language requires advanced reasoning capabilities, approaching those of Artificial General Intelligence (AGI). However, the evaluation remains challenging, as perfectly representing reality is inherently elusive, and traditional methods like manual or direct comparison of mathematical statements (Ramamonjison et al., 2023) are insufficient for assessing true modeling ability. We propose a process-oriented framework to evaluate LLMs' ability to construct mathematical models, using solvers to compare outputs with ground truth. Introducing Mamo, a benchmark with 1,209 questions covering ordinary differential equations, linear programming, and mixed-integer linear programming, we enable automatic evaluation of modeling accuracy. The results show that existing LLMs struggle with complex mathematical modeling tasks, with larger models demonstrating superior performance, while open-source models remain competitive in simpler cases but still fall short of proprietary models in more challenging problems.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 15 of its 36 references that are in the map. First posted on arXiv 21 May 2024.

Builds on

15 of 36 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoning5 Mar
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models5 Feb
  3. 2023Evaluating and Improving Tool-Augmented Computation-Intensive Math Reasoning4 Jun
  4. 2023How well do Large Language Models perform in Arithmetic tasks?16 Mar
  5. 2022Solving Quantitative Reasoning Problems with Language Models29 Jun

Other references · 10

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  5. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  6. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  7. 2023NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptionscited by 133
  8. 2023Solving Math Word Problems by Combining Language Models With Symbolic Solverscited by 124
  9. 2024MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmarkcited by 103
  10. 2023Language Models can be Logical Solverscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 12; 2026: 31.

46 papers.

Sort:

By area:

  1. 2024LLMOPT: Learning to Define and Solve General Optimization Problems from Scratchcited by 102
  2. 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
  3. 2024BPP-Search: Enhancing Tree of Thought Reasoning for Mathematical Modeling Problem Solvingcited by 15
  4. 2025A Survey on Mathematical Reasoning and Optimization with Large Language Modelscited by 12
  5. 2026Constructing Industrial-Scale Optimization Modeling Benchmarkcited by 10
  6. 2025A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solvingcited by 9
  7. 2025SAC-Opt: Semantic Anchors for Iterative Correction in Optimization Modelingcited by 8
  8. 2025VeriEquivBench: An Equivalence Score for Ground-Truth-Free Evaluation of Formally Verifiable Codecited by 7
  9. 2026ConstraintBench: Benchmarking LLM Constraint Reasoning on Direct Optimizationcited by 6
  10. 2026OPT-Engine: Benchmarking the Limits of LLMs in Optimization Modeling via Complexity Scalingcited by 5
  11. 2026NEMO: Execution-Aware Optimization Modeling via Autonomous Coding Agentscited by 5
  12. 2025ModelingAgent: Bridging LLMs and Mathematical Modeling for Real-World Challengescited by 4
34 more
  1. 2025Large Language Models in Operations Research: Methods, Applications, and Challengescited by 4
  2. 2026FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimizationcited by 4
  3. 2025CALM Before the STORM: Unlocking Native Reasoning for Optimization Modelingcited by 3
  4. 2026OptiVerse: A Comprehensive Benchmark towards Optimization Problem Solvingcited by 3
  5. 2026Execution-Verified Reinforcement Learning for Optimization Modelingcited by 2
  6. 2026TriVAL: A Tri-Validation Framework for Faithful Automatic Optimization Modelingcited by 2
  7. 2026OR-Space: A Full-Lifecycle Workspace Benchmark for Industrial Optimization Agentscited by 2
  8. 2026OptSkills: Learning Generalizable Optimization Skills from Problem Archetypes via Cluster-Based Distillationcited by 2
  9. 2026Beyond Objective Equivalence: Constraint Injection for LLM-Based Optimization Modeling on Vehicle Routing Problemscited by 2
  10. 2025Teaching LLMs to Think Mathematically: A Critical Study of Decision-Making via Optimizationcited by 1
  11. 2025Can Large Language Models Autoformalize Kinematics?cited by 1
  12. 2025SOCRATES: Simulation Optimization with Correlated Replicas and Adaptive Trajectory Evaluationscited by 1
  13. 2025Large Language Model-Assisted Planning of Electric Vehicle Charging Infrastructure with Real-World Case Studycited by 1
  14. 2026MiniOpt: Reasoning to Model and Solve General Optimization Problems with Limited Resourcescited by 1
  15. 2026SCHEDBench: A Benchmark for Evaluating LLM Constraint Faithfulness in Natural-Language Combinatorial Schedulingcited by 1
  16. 2026AlgoWorlds: Benchmarking Tool Use for Global Optimization in Algorithmic Worldscited by 1
  17. 2025Step-Opt: Boosting Optimization Modeling in LLMs through Iterative Data Synthesis and Structured Validationcited by 0
  18. 2026Dual-Cluster Memory Agent: Resolving Multi-Paradigm Ambiguity in Optimization Problem Solvingcited by 0
  19. 2026Models Can Model, But Can't Bind: Structured Grounding in Text-to-Optimizationcited by 0
  20. 2026ORAgentBench: Can LLM Agents Solve Challenging Operations Research Tasks End to End?cited by 0
  21. 2026A2utoLPBench: An Auto-Generated, Agent-Friendly LP Benchmark via Inverse-KKT Constructioncited by 0
  22. 2026JOR-Bench: Japanese Operations Research Benchmarks for Large Language Modelscited by 0
  23. 2026LM-GRASP: Instance-Specific Language Models for Combinatorial Construction via Online Imitation Learningcited by 0
  24. 2026BBOWP-Bench: Evaluating LLMs on Black-Box Optimization Word Problemscited by 0
  25. 2026IR2Solve: Structured Intermediate Representations for Cost-Efficient Optimization Autoformulationcited by 0
  26. 2026Large language models for partial differential equation workflowscited by 0
  27. 2026AI Control Scientist: LLM-driven Agentic System for Automated Control Designcited by 0
  28. 2026SOVER: Formal Certification of Optimization Reformulations via LLM-Assisted SMT Verificationcited by 0
  29. 2026Adapting to Evolving Requirements: Agentic AI for Retail Supply Chain Operationscited by 0
  30. 2026little m: An AI Agent for Industrial Process Optimizationcited by 0
  31. 2026LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimizationcited by 0
  32. 2026SemOPT: Fixing Semantic Errors in LLM-based Optimization Modeling via Reward-Guided Searchcited by 0
  33. 2026Right Answers, Costly Models: The Efficiency Gap in LLM-based Optimization Modelingcited by 0
  34. 2026OR for AI That Does OR: Routing LLMs up the Escalator inside the OSCAR Frameworkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  2. 2025Can LLMs textitunderstand Math? -- Exploring the Pitfalls in Mathematical Reasoning
  3. 2025Large Language Models and Mathematical Reasoning Failures
  4. 2024Large Language Models for Mathematical Reasoning: Progresses and Challenges
  5. 2025Evaluation of LLMs for mathematical problem solving
  6. 2025Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approach
  7. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges
  8. 2025Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges