mlsanityData to 14 Oct 2026

Preference-Driven Multi-Objective Combinatorial Optimization with Conditional Computation

arXiv 2506.08898 ↗First posted 10 Jun 2025cs.AIcited by 6

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Recent deep reinforcement learning methods have achieved remarkable success in solving multi-objective combinatorial optimization problems (MOCOPs) by decomposing them into multiple subproblems, each associated with a specific weight vector. However, these methods typically treat all subproblems equally and solve them using a single model, hindering the effective exploration of the solution space and thus leading to suboptimal performance. To overcome the limitation, we propose POCCO, a novel plug-and-play framework that enables adaptive selection of model structures for subproblems, which are subsequently optimized based on preference signals rather than explicit reward values. Specifically, we design a conditional computation block that routes subproblems to specialized neural architectures. Moreover, we propose a preference-driven optimization algorithm that learns pairwise preferences between winning and losing solutions. We evaluate the efficacy and versatility of POCCO by applying it to two state-of-the-art neural methods for MOCOPs. Experimental results across four classic MOCOP benchmarks demonstrate its significant superiority and strong generalization.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 31 of its 70 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

31 of 70 references.

References · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  4. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  5. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
  6. 2015Pointer Networkscited by 895
  7. 2016Neural Combinatorial Optimization with Reinforcement Learningcited by 608
  8. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  9. 2020POMO: Policy Optimization with Multiple Optima for Reinforcement Learningcited by 348
  10. 2018Reinforcement Learning for Solving the Vehicle Routing Problemcited by 333
  11. 2024Mixture-of-Depths: Dynamically allocating compute in transformer-based language modelscited by 258
  12. 2018Learning to Perform Local Rewriting for Combinatorial Optimizationcited by 185
19 more
  1. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
  2. 2019Learning Improvement Heuristics for Solving Routing Problemscited by 164
  3. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 148
  4. 2021Learning to Delegate for Large-scale Vehicle Routingcited by 109
  5. 2023Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Losscited by 108
  6. 2021Efficient Active Search for Combinatorial Optimization Problemscited by 100
  7. 2019Deep Reinforcement Learning for Multi-objective Optimizationcited by 90
  8. 2021Learning Collaborative Policies to Solve NP-hard Routing Problemscited by 86
  9. 2023RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmarkcited by 71
  10. 2024MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Expertscited by 70
  11. 2023Towards Omni-generalizable Neural Methods for Vehicle Routing Problemscited by 69
  12. 2022Pareto Set Learning for Neural Multi-objective Combinatorial Optimizationcited by 56
  13. 2024Learning to Handle Complex Constraints for Vehicle Routing Problemscited by 29
  14. 2023Efficient Meta Neural Heuristic for Multi-Objective Combinatorial Optimizationcited by 21
  15. 2021Meta-Learning-Based Deep Reinforcement Learning for Multiobjective Optimization Problemscited by 20
  16. 2024SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixturecited by 19
  17. 2025BOPO: Neural Combinatorial Optimization via Best-anchored and Objective-guided Preference Optimizationcited by 14
  18. 2020MODRL/D-AM: Multiobjective Deep Reinforcement Learning Algorithm Using Decomposition and Attention Model for Multiobjective Optimizationcited by 11
  19. 2021MODRL/D-EL: Multiobjective Deep Reinforcement Learning with Evolutionary Learning for Multiobjective Optimizationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Adversarial Instance Generation and Robust Training for Neural Combinatorial Optimization with Multiple Objectives
  2. 2024Leader Reward for POMO-Based Neural Combinatorial Optimization
  3. 2025Preference Optimization for Combinatorial Optimization Problems
  4. 2025Graph-Supported Dynamic Algorithm Configuration for Multi-Objective Combinatorial Optimization
  5. 2026Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimization
  6. 2026Rethinking Efficiency in Neural Combinatorial Optimization: Batched Preference Optimization with Mamba
  7. 2026HyCO: A Hybrid Neural Solver for Combinatorial Optimization
  8. 2025ASAP: Exploiting the Satisficing Generalization Edge in Neural Combinatorial Optimization