# Auto-Formulating Dynamic Programming Problems with Large Language Models

arXiv: https://arxiv.org/abs/2507.11737 · HTML: https://mlsanity.com/paper/2507.11737/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 35 references that are in the map.
- First posted on arXiv 15 Jul 2025.

## Builds on (most cited first, 21 of 21)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Deep Learning for Symbolic Mathematics](https://mlsanity.com/paper/1912.01412/) (2019)
- [Secrets of RLHF in Large Language Models Part I: PPO](https://mlsanity.com/paper/2307.04964/) (2023)
- [NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions](https://mlsanity.com/paper/2303.08233/) (2023)
- [ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling](https://mlsanity.com/paper/2405.17743/) (2024)
- [LLM Generated Persona is a Promise with a Catch](https://mlsanity.com/paper/2503.16527/) (2025)
- [MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes](https://mlsanity.com/paper/2412.19260/) (2024)
- [OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling](https://mlsanity.com/paper/2407.09887/) (2024)
- [OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling](https://mlsanity.com/paper/2502.11102/) (2025)
- [Global Lyapunov functions: a long-standing open problem in mathematics, with symbolic transformers](https://mlsanity.com/paper/2410.08304/) (2024)
- [Large Language Models for Market Research: A Data-augmentation Approach](https://mlsanity.com/paper/2412.19363/) (2024)
- [Online Scheduling for LLM Inference with KV Cache Constraints](https://mlsanity.com/paper/2502.07115/) (2025)
- [Optimizing LLM Inference: Fluid-Guided Online Scheduling with Memory Constraints](https://mlsanity.com/paper/2504.11320/) (2025)
- [LOLA: LLM-Assisted Online Learning Algorithm for Content Experiments](https://mlsanity.com/paper/2406.02611/) (2024)
- [Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents](https://mlsanity.com/paper/2504.07347/) (2025)
- [Robust and Adaptive Optimization under a Large Language Model Lens](https://mlsanity.com/paper/2501.00568/) (2024)
- [A Tutorial on Teaching Data Analytics with Generative AI](https://mlsanity.com/paper/2411.07244/) (2024)

## Built on by (most cited first, 16 of 16)

- [StepORLM: A Self-Evolving Framework With Generative Process Supervision For Operations Research Language Models](https://mlsanity.com/paper/2509.22558/) (2025)
- [A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving](https://mlsanity.com/paper/2509.08269/) (2025)
- [AI Agents for Inventory Control: Human-LLM-OR Complementarity](https://mlsanity.com/paper/2602.12631/) (2026)
- [ORThought: Benchmarking and Automating Logistics Optimization Modeling](https://mlsanity.com/paper/2508.14410/) (2025)
- [Ask, Clarify, Optimize: Human-LLM Agent Collaboration for Smarter Inventory Control](https://mlsanity.com/paper/2601.00121/) (2025)
- [OptiRepair: Closed-Loop Diagnosis and Repair of Supply Chain Optimization Models with LLM Agents](https://mlsanity.com/paper/2602.19439/) (2026)
- [LLM-SAA: LLM-persona Generated Distributions for Decision-making](https://mlsanity.com/paper/2602.06357/) (2026)
- [Canonical Intermediate Representation for LLM-based optimization problem formulation and code generation](https://mlsanity.com/paper/2602.02029/) (2026)
- [Democratizing Large-Scale Re-Optimization with LLM-Guided Model Patches](https://mlsanity.com/paper/2605.18692/) (2026)
- [OR-Space: A Full-Lifecycle Workspace Benchmark for Industrial Optimization Agents](https://mlsanity.com/paper/2605.28158/) (2026)
- [Optimizing Service Operations via LLM-Powered Multi-Agent Simulation](https://mlsanity.com/paper/2604.04383/) (2026)
- [ORPR: An OR-Guided Pretrain-then-Reinforce Learning Model for Inventory Management](https://mlsanity.com/paper/2512.19001/) (2025)
- [The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms](https://mlsanity.com/paper/2606.25450/) (2026)
- [COOPA: A Modular LLM Agent Architecture for Operations Research Problems](https://mlsanity.com/paper/2606.27611/) (2026)
- [LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimization](https://mlsanity.com/paper/2609.34427/) (2026)
- [OR for AI That Does OR: Routing LLMs up the Escalator inside the OSCAR Framework](https://mlsanity.com/paper/2610.00912/) (2026)

## Abstract (arXiv)

Dynamic programming (DP) is a fundamental method in operations research, but formulating DP models has traditionally required expert knowledge of both the problem context and DP techniques. Large Language Models (LLMs) offer the potential to automate this process. However, DP problems pose unique challenges due to their inherently stochastic transitions and the limited availability of training data. These factors make it difficult to directly apply existing LLM-based models or frameworks developed for other optimization problems, such as linear or integer programming. We introduce DP-Bench, the first benchmark covering a wide range of textbook-level DP problems to enable systematic evaluation. We present Dynamic Programming Language Model (DPLM), a 7B-parameter specialized model that achieves performance comparable to state-of-the-art LLMs like OpenAI's o1 and DeepSeek-R1, and surpasses them on hard problems. Central to DPLM's effectiveness is DualReflect, our novel synthetic data generation pipeline, designed to scale up training data from a limited set of initial examples. DualReflect combines forward generation for diversity and backward generation for reliability. Our results reveal a key insight: backward generation is favored in low-data regimes for its strong correctness guarantees, while forward generation, though lacking such guarantees, becomes increasingly valuable at scale for introducing diverse formulations. This trade-off highlights the complementary strengths of both approaches and the importance of combining them.