mlsanityData to 7 Oct 2026

SORREL: Suboptimal-Demonstration-Guided Reinforcement Learning for Learning to Branch

arXiv 2412.15534 ↗First posted 20 Dec 2024cs.LGcited by 9

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Mixed Integer Linear Program (MILP) solvers are mostly built upon a Branch-and-Bound (B&B) algorithm, where the efficiency of traditional solvers heavily depends on hand-crafted heuristics for branching. The past few years have witnessed the increasing popularity of data-driven approaches to automatically learn these heuristics. However, the success of these methods is highly dependent on the availability of high-quality demonstrations, which requires either the development of near-optimal heuristics or a time-consuming sampling process. This paper averts this challenge by proposing Suboptimal-Demonstration-Guided Reinforcement Learning (SORREL) for learning to branch. SORREL selectively learns from suboptimal demonstrations based on value estimation. It utilizes suboptimal demonstrations through both offline reinforcement learning on the demonstrations generated by suboptimal heuristics and self-imitation learning on past good experiences sampled by itself. Our experiments demonstrate its advanced performance in both branching quality and training efficiency over previous methods for various MILPs.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 29 of its 47 references that are in the map. First posted on arXiv 20 Dec 2024.

Builds on

29 of 47 references.

References · 29

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  3. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  4. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  5. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 612
  6. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  7. 2019Exact Combinatorial Optimization with Graph Convolutional Neural Networkscited by 313
  8. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  9. 2020Solving Mixed Integer Programs Using Neural Networkscited by 178
  10. 2018Self-Imitation Learningcited by 135
  11. 2019Reinforcement Learning for Integer Programming: Learning to Cutcited by 108
  12. 2020Hybrid Models for Learning to Branchcited by 95
17 more
  1. 2020Parameterizing Branch-and-Bound Search Trees to Learn Branching Policiescited by 62
  2. 2022Learning To Cut By Looking Ahead: Cutting Plane Selection via Imitation Learningcited by 58
  3. 2021Learning to Schedule Heuristics in Branch-and-Boundcited by 45
  4. 2022Learning to branch with Tree MDPscited by 45
  5. 2023Learning Cut Selection for Mixed-Integer Linear Programming via Hierarchical Sequence Modelcited by 44
  6. 2021Learning a Large Neighborhood Search Algorithm for Mixed Integer Programscited by 41
  7. 2021MLGO: a Machine Learning Guided Compiler Optimizations Frameworkcited by 40
  8. 2023Searching Large Neighborhoods for Integer Linear Programs with Contrastive Learningcited by 37
  9. 2021Learning Large Neighborhood Search Policy for Integer Programmingcited by 35
  10. 2020Reinforcement Learning for Variable Selection in a Branch and Bound Algorithmcited by 28
  11. 2022Lookback for Learning to Branchcited by 26
  12. 2022Reinforcement Learning for Branch-and-Bound Optimisation using Retrospective Trajectoriescited by 19
  13. 2022An Improved Reinforcement Learning Algorithm for Learning to Branchcited by 14
  14. 2018Learning to Search via Retrospective Imitationcited by 13
  15. 2022Deep Reinforcement Learning for Exact Combinatorial Optimization: Learning to Branchcited by 12
  16. 2018Cuts, Primal Heuristics, and Learning to Branch for the Time-Dependent Traveling Salesman Problemcited by 11
  17. 2022Branch Ranking for Efficient Mixed-Integer Programming via Offline Ranking-based Policy Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 5.

9 papers.

  1. 2025Regularized Langevin Dynamics for Combinatorial Optimizationcited by 10
  2. 2025FrontierCO: Real-World and Large-Scale Evaluation of Machine Learning Solvers for Combinatorial Optimizationcited by 10
  3. 2026LLM4Branch: Large Language Model for Discovering Efficient Branching Policies of Integer Programscited by 4
  4. 2025Dynamic Stratified Contrastive Learning with Upstream Augmentation for MILP Branchingcited by 1
  5. 2026A General Neural Backbone for Mixed-Integer Linear Optimization via Dual Attentioncited by 1
  6. 2025One4Many-StablePacker: An Efficient Deep Reinforcement Learning Framework for the 3D Bin Packing Problemcited by 0
  7. 2026Towards Solving Polynomial-Objective Integer Programming with Hypergraph Neural Networkscited by 0
  8. 2026Solving Integer Linear Programming with Parallel Temperingcited by 0
  9. 2026Autoresearch in Mixed-Integer Linear and Nonlinear Programmingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning Branching Policies for MILPs with Proximal Policy Optimization
  2. 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regression
  3. 2024CAMBranch: Contrastive Learning with Augmented MILPs for Branching
  4. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review
  5. 2026Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learning
  6. 2026Teaching LLMs to Generate Challenging MILP Instances via Solver Feedback
  7. 2019Reinforcement Learning from Imperfect Demonstrations under Soft Expert Guidance
  8. 2025Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration