Preference-Driven Multi-Objective Combinatorial Optimization with Conditional Computation
On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization
Abstract (arXiv)
Recent deep reinforcement learning methods have achieved remarkable success in solving multi-objective combinatorial optimization problems (MOCOPs) by decomposing them into multiple subproblems, each associated with a specific weight vector. However, these methods typically treat all subproblems equally and solve them using a single model, hindering the effective exploration of the solution space and thus leading to suboptimal performance. To overcome the limitation, we propose POCCO, a novel plug-and-play framework that enables adaptive selection of model structures for subproblems, which are subsequently optimized based on preference signals rather than explicit reward values. Specifically, we design a conditional computation block that routes subproblems to specialized neural architectures. Moreover, we propose a preference-driven optimization algorithm that learns pairwise preferences between winning and losing solutions. We evaluate the efficacy and versatility of POCCO by applying it to two state-of-the-art neural methods for MOCOPs. Experimental results across four classic MOCOP benchmarks demonstrate its significant superiority and strong generalization.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 31 of its 70 references that are in the map. First posted on arXiv 10 Jun 2025.
Builds on
31 of 70 references.
References · 31
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
- 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
- 2015Pointer Networkscited by 895
- 2016Neural Combinatorial Optimization with Reinforcement Learningcited by 608
- 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
- 2020POMO: Policy Optimization with Multiple Optima for Reinforcement Learningcited by 348
- 2018Reinforcement Learning for Solving the Vehicle Routing Problemcited by 333
- 2024Mixture-of-Depths: Dynamically allocating compute in transformer-based language modelscited by 258
- 2018Learning to Perform Local Rewriting for Combinatorial Optimizationcited by 185
19 more
- 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
- 2019Learning Improvement Heuristics for Solving Routing Problemscited by 164
- 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 148
- 2021Learning to Delegate for Large-scale Vehicle Routingcited by 109
- 2023Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Losscited by 108
- 2021Efficient Active Search for Combinatorial Optimization Problemscited by 100
- 2019Deep Reinforcement Learning for Multi-objective Optimizationcited by 90
- 2021Learning Collaborative Policies to Solve NP-hard Routing Problemscited by 86
- 2023RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmarkcited by 71
- 2024MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Expertscited by 70
- 2023Towards Omni-generalizable Neural Methods for Vehicle Routing Problemscited by 69
- 2022Pareto Set Learning for Neural Multi-objective Combinatorial Optimizationcited by 56
- 2024Learning to Handle Complex Constraints for Vehicle Routing Problemscited by 29
- 2023Efficient Meta Neural Heuristic for Multi-Objective Combinatorial Optimizationcited by 21
- 2021Meta-Learning-Based Deep Reinforcement Learning for Multiobjective Optimization Problemscited by 20
- 2024SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixturecited by 19
- 2025BOPO: Neural Combinatorial Optimization via Best-anchored and Objective-guided Preference Optimizationcited by 14
- 2020MODRL/D-AM: Multiobjective Deep Reinforcement Learning Algorithm Using Decomposition and Attention Model for Multiobjective Optimizationcited by 11
- 2021MODRL/D-EL: Multiobjective Deep Reinforcement Learning with Evolutionary Learning for Multiobjective Optimizationcited by 8
Built on by
Citing papers per year: 2025: 1; 2026: 5.
6 papers.
- 2026Towards Efficient Constraint Handling in Neural Solvers for Routing Problemscited by 3
- 2025Beyond Simple Graphs: Neural Multi-Objective Routing on Multigraphscited by 2
- 2026Hindsight Preference Replay Improves Preference-Conditioned Multi-Objective Reinforcement Learningcited by 0
- 2026CAMO: A Conditional Neural Solver for the Multi-objective Multiple Traveling Salesman Problemcited by 0
- 2026WeCon: An Efficient Weight-Conditioned Neural Solver for Multi-Objective Combinatorial Optimization Problemscited by 0
- 2026AutoPref: Automatic Discovery of Task-Specific Preference Objectives for Neural Combinatorial Optimizationcited by 0
- 2026AutoPref: Automatic Discovery of Task-Specific Preference Objectives for Neural Combinatorial Optimization30 Jul 2026
- 2026WeCon: An Efficient Weight-Conditioned Neural Solver for Multi-Objective Combinatorial Optimization Problems20 May 2026
- 2026CAMO: A Conditional Neural Solver for the Multi-objective Multiple Traveling Salesman Problem19 Mar 2026
- 2026Towards Efficient Constraint Handling in Neural Solvers for Routing Problems17 Feb 2026
- 2026Hindsight Preference Replay Improves Preference-Conditioned Multi-Objective Reinforcement Learning8 Jan 2026
- 2025Beyond Simple Graphs: Neural Multi-Objective Routing on Multigraphs27 Jun 2025
Similar, not cited
Close in wording, no citation either way
- 2026Adversarial Instance Generation and Robust Training for Neural Combinatorial Optimization with Multiple Objectives
- 2024Leader Reward for POMO-Based Neural Combinatorial Optimization
- 2025Preference Optimization for Combinatorial Optimization Problems
- 2025Graph-Supported Dynamic Algorithm Configuration for Multi-Objective Combinatorial Optimization
- 2026Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimization
- 2026Rethinking Efficiency in Neural Combinatorial Optimization: Batched Preference Optimization with Mamba
- 2026HyCO: A Hybrid Neural Solver for Combinatorial Optimization
- 2025ASAP: Exploiting the Satisficing Generalization Edge in Neural Combinatorial Optimization