mlsanityData to 14 Oct 2026

CORL: Reinforcement Learning of MILP Policies Solved via Branch and Bound

arXiv 2512.11169 ↗First posted 11 Dec 2025cs.AI cs.LG cs.SYcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Combinatorial sequential decision making problems are typically modeled as mixed integer linear programs (MILPs) and solved via branch and bound (B&B) algorithms. The inherent difficulty of modeling MILPs that accurately represent stochastic real world problems leads to suboptimal performance in the real world. Recently, machine learning methods have been applied to build MILP models for decision quality rather than how accurately they model the real world problem. However, these approaches typically rely on supervised learning, assume access to true optimal decisions, and use surrogates for the MILP gradients. In this work, we introduce a proof of concept CORL framework that end to end fine tunes an MILP scheme using reinforcement learning (RL) on real world data to maximize its operational performance. We enable this by casting an MILP solved by B&B as a differentiable stochastic policy compatible with RL. We validate the CORL method in a simple illustrative combinatorial sequential decision making example.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 28 of its 42 references that are in the map. First posted on arXiv 11 Dec 2025.

Builds on

28 of 42 references.

References · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  4. 2016Neural Combinatorial Optimization with Reinforcement Learningcited by 608
  5. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  6. 2017Smart "Predict, then Optimize"cited by 339
  7. 2020Reinforcement Learning for Combinatorial Optimization: A Surveycited by 236
  8. 2018Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimizationcited by 227
  9. 2019Differentiation of Blackbox Combinatorial Solverscited by 226
  10. 2021End-to-End Constrained Optimization Learning: A Surveycited by 157
  11. 2023Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunitiescited by 130
  12. 2019MIPaaL: Mixed Integer Program as a Layercited by 94
16 more
  1. 2023A Survey of Contextual Optimization Methods for Decision Making under Uncertaintycited by 82
  2. 2019Combinatorial Optimization by Graph Pointer Networks and Hierarchical Reinforcement Learningcited by 78
  3. 2021Decision-Focused Learning: Through the Lens of Learning to Rankcited by 62
  4. 2021Proper Value Equivalencecited by 33
  5. 2022Value Gradient weighted Model-Based Reinforcement Learningcited by 32
  6. 2019The Differentiable Cross-Entropy Methodcited by 27
  7. 2025Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classificationcited by 26
  8. 2020Reinforcement Learning for Mixed-Integer Problems Based on MPCcited by 12
  9. 2023A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learningcited by 9
  10. 2021Smart Feasibility Pump: Reinforcement Learning for (Mixed) Integer Programmingcited by 8
  11. 2023Operational Research: Methods and Applicationscited by 6
  12. 2024DiffTORI: Differentiable Trajectory Optimization for Deep Reinforcement and Imitation Learningcited by 6
  13. 2025Reinforcement learning with combinatorial actions for coupled restless banditscited by 5
  14. 2024Optimality Conditions for Model Predictive Control: Rethinking Predictive Model Designcited by 4
  15. 2025All AI Models are Wrong, but Some are Optimalcited by 4
  16. 2025Bi-Level Reinforcement Learning Pathway for Sim-to-Real Optimalitycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Markov Decision Process for Variable Selection in Branch & Bound
  2. 2025Planning in Branch-and-Bound: Model-Based Reinforcement Learning for Exact Combinatorial Optimization
  3. 2025Heuristics for Combinatorial Optimization via Value-based Reinforcement Learning: A Unified Framework and Analysis
  4. 2019Co-training for Policy Learning
  5. 2025Learning Branching Policies for MILPs with Proximal Policy Optimization
  6. 2025Realizable Abstractions: Near-Optimal Hierarchical Reinforcement Learning
  7. 2025Solve Smart, Not Often: Policy Learning for Costly MILP Re-solving
  8. 2022Reinforcement Learning for Branch-and-Bound Optimisation using Retrospective Trajectories