mlsanityData to 7 Oct 2026

Towards Generalization-Oriented Models for Vehicle Routing Problems with Mixture-of-Experts

arXiv 2605.26776 ↗First posted 26 May 2026cs.LG cs.AIcited by 0

On the map: Mixture-of-experts → MoE in reinforcement learning and robotics

Abstract (arXiv)

In recent years, Deep Reinforcement Learning (DRL) has achieved substantial progress on Vehicle Routing Problems (VRPs). However, existing DRL-based methods are typically trained on instances generated from a uniform distribution, which limits their performance under real-world distribution shifts. In this paper, we aim to develop a generalization-oriented model that partitions the policy network into multiple modules and adaptively recombines modules to form specific policies during inference. Specifically, we propose Residual Refined Experts with Instance-level Gating (R2E-IG) to improve cross-distribution generalization. Our contributions are threefold: (1) We introduce a Residual Refined Expert (R2E) architecture that enhance expert expressiveness via residual refinement; (2) We design an instance-level gating mechanism that learns distribution-aware instance representations and routes inputs to suitable modules; (3) We propose a mixed-distribution training mechanism equipped with Dynamic Weight Adaption (DWA), which dynamically reweights training data from different distributions to emphasize more informative ones. Extensive experiments show that R2E-IG achieves competitive performance against state-of-the-art baselines on both in-distribution and out-of-distribution instances across synthetic and benchmark datasets. Moreover, R2E-IG is generic and can be easily integrated into existing DRL-based methods to further improve performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MoE in reinforcement learning and robotics, one of 12 directions in Mixture-of-experts. Builds on 43 of its 65 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

43 of 65 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  7. 2015Pointer Networkscited by 882
  8. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 612
  9. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  10. 2016Neural Combinatorial Optimization with Reinforcement Learningcited by 600
  11. 2017Learning Combinatorial Optimization Algorithms over Graphscited by 557
  12. 2020POMO: Policy Optimization with Multiple Optima for Reinforcement Learningcited by 341
31 more
  1. 2018Reinforcement Learning for Solving the Vehicle Routing Problemcited by 327
  2. 2021BASE Layers: Simplifying Training of Large, Sparse Modelscited by 315
  3. 2019An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problemcited by 265
  4. 2023From Sparse to Soft Mixtures of Expertscited by 245
  5. 2022Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Expertscited by 202
  6. 2024OpenMoE: An Early Effort on Open Mixture-of-Experts Language Modelscited by 185
  7. 2020Generalize a Small Pre-trained Model to Arbitrarily Large TSP Instancescited by 161
  8. 2023Neural Combinatorial Optimization with Heavy Decoder: Toward Large Scale Generalizationcited by 140
  9. 2020Learning to Dispatch for Job Shop Scheduling via Deep Reinforcement Learningcited by 139
  10. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  11. 2022Sym-NCO: Leveraging Symmetricity for Neural Combinatorial Optimizationcited by 118
  12. 2019Neural Large Neighborhood Search for the Capacitated Vehicle Routing Problemcited by 117
  13. 2021NeuroLKH: Combining Deep Learning Model with Lin-Kernighan-Helsgaun Heuristic for Solving the Traveling Salesman Problemcited by 102
  14. 2020Learning 2-opt Heuristics for the Traveling Salesman Problem via Deep Reinforcement Learningcited by 100
  15. 2020Learning the Travelling Salesperson Problem Requires Rethinking Generalizationcited by 96
  16. 2021Deep Policy Dynamic Programming for Vehicle Routing Problemscited by 94
  17. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91
  18. 2022Simulation-guided Beam Search for Neural Combinatorial Optimizationcited by 84
  19. 2023Learning to Search Feasible and Infeasible Regions of Routing Problems with Flexible Neural k-Optcited by 81
  20. 2021Graph Neural Network Guided Local Search for the Traveling Salesperson Problemcited by 70
  21. 2024MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Expertscited by 69
  22. 2023Towards Omni-generalizable Neural Methods for Vehicle Routing Problemscited by 68
  23. 2022Learning Generalizable Models for Vehicle Routing Problems via Knowledge Distillationcited by 61
  24. 2023Towards Generalizable Neural Solvers for Vehicle Routing Problems via Ensemble with Transferrable Local Policycited by 61
  25. 2024Multi-Task Learning for Routing Problem with Cross-Problem Zero-Shot Generalizationcited by 49
  26. 2022Learning to Solve Routing Problems via Distributionally Robust Optimizationcited by 31
  27. 2021EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gatecited by 30
  28. 2021SpeechMoE2: Mixture-of-Experts Model with Improved Routingcited by 26
  29. 2022On the Generalization of Neural Combinatorial Optimization Heuristicscited by 24
  30. 2023Meta-SAGE: Scale Meta-Learning Scheduled Adaptation with Guided Exploration for Mitigating Scale Shift on Combinatorial Optimizationcited by 23
  31. 2025An End-to-End Learning Approach for Solving Capacitated Location-Routing Problemscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving Cross-Problem Vehicle Routing with Locally Augmented Preferences and Representation Disentanglement
  2. 2026Reinforcement Learning Enhanced LLM Agents for Complex Vehicle Routing Problems
  3. 2018Deep Mixture of Experts via Shallow Embedding
  4. 2026Constraints Matrix Diffusion based Generative Neural Solver for Vehicle Routing Problems
  5. 2026MoE-GRPO: Optimizing Mixture-of-Experts via Reinforcement Learning in Vision-Language Models
  6. 2024Generalizing Motion Planners with Mixture of Experts for Autonomous Driving
  7. 2026Two Black Boxes, One Solver: Encoder Probing and Decoder Attribution for Neural Multi-Attribute VRP under Hard-Mask and Recourse Decoders
  8. 2025State Combinatorial Generalization In Decision Making With Conditional Diffusion Models