mlsanityData to 7 Oct 2026

Symmetric Replay Training: Enhancing Sample Efficiency in Deep Reinforcement Learning for Combinatorial Optimization

arXiv 2306.01276 ↗First posted 2 Jun 2023cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) has significantly advanced the field of combinatorial optimization (CO). However, its practicality is hindered by the necessity for a large number of reward evaluations, especially in scenarios involving computationally intensive function assessments. To enhance the sample efficiency, we propose a simple but effective method, called symmetric replay training (SRT), which can be easily integrated into various DRL methods. Our method leverages high-reward samples to encourage exploration of the under-explored symmetric regions without additional online interactions - free. Through replay training, the policy is trained to maximize the likelihood of the symmetric trajectories of discovered high-rewarded samples. Experimental results demonstrate the consistent improvement of our method in sample efficiency across diverse DRL methods applied to real-world tasks, such as molecular optimization and hardware design.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 59 of its 92 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

59 of 92 references.

References · 59

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2016Group Equivariant Convolutional Networkscited by 1,154
  5. 2021E(n) Equivariant Graph Neural Networkscited by 951
  6. 2015Pointer Networkscited by 882
  7. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  8. 2018Attention, Learn to Solve Routing Problems!cited by 646
  9. 2016Neural Combinatorial Optimization with Reinforcement Learningcited by 600
  10. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  11. 2017Learning Combinatorial Optimization Algorithms over Graphscited by 557
  12. 2020Reinforcement Learning with Augmented Datacited by 452
38 more
  1. 2019General E(2)-Equivariant Steerable CNNscited by 441
  2. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  3. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  4. 2020POMO: Policy Optimization with Multiple Optima for Reinforcement Learningcited by 341
  5. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  6. 2021GFlowNet Foundationscited by 286
  7. 2017Molecular De Novo Design through Deep Reinforcement Learningcited by 264
  8. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  9. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  10. 2021SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimizationcited by 229
  11. 2016Sample Efficient Actor-Critic with Experience Replaycited by 185
  12. 2018Learning to Perform Local Rewriting for Combinatorial Optimizationcited by 183
  13. 2022Biological Sequence Design with GFlowNetscited by 172
  14. 2020Revisiting Fundamentals of Experience Replaycited by 171
  15. 2022Sample Efficiency Matters: A Benchmark for Practical Molecular Optimizationcited by 151
  16. 2020Learning to Dispatch for Job Shop Scheduling via Deep Reinforcement Learningcited by 139
  17. 2020MDP Homomorphic Networks: Group Symmetries in Reinforcement Learningcited by 133
  18. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 129
  19. 2022DIMES: A Differentiable Meta Solver for Combinatorial Optimization Problemscited by 127
  20. 2022Sym-NCO: Leveraging Symmetricity for Neural Combinatorial Optimizationcited by 118
  21. 2019Neural Large Neighborhood Search for the Capacitated Vehicle Routing Problemcited by 117
  22. 2021Learning to Delegate for Large-scale Vehicle Routingcited by 108
  23. 2021NeuroLKH: Combining Deep Learning Model with Lin-Kernighan-Helsgaun Heuristic for Solving the Traveling Salesman Problemcited by 102
  24. 2018Multi-Objective De Novo Drug Design with Conditional Graph Generative Modelcited by 101
  25. 2020Multi-Decoder Attention Model with Embedding Glimpse for Solving Vehicle Routing Problemscited by 100
  26. 2021Efficient Active Search for Combinatorial Optimization Problemscited by 99
  27. 2021Matrix Encoding Networks for Neural Combinatorial Optimizationcited by 89
  28. 2022Simulation-guided Beam Search for Neural Combinatorial Optimizationcited by 84
  29. 2023Better Training of GFlowNets with Local Credit and Incomplete Trajectoriescited by 83
  30. 2022SO(2)-Equivariant Reinforcement Learningcited by 79
  31. 2019Combinatorial Bayesian Optimization using the Graph Cartesian Productcited by 78
  32. 2019ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendationscited by 77
  33. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 76
  34. 2023Towards Understanding and Improving GFlowNet Trainingcited by 74
  35. 2018SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translationcited by 72
  36. 2020A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generationcited by 72
  37. 2018Bayesian Optimization of Combinatorial Structurescited by 64
  38. 2020Learning What to Defer for Maximum Independent Setscited by 53

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 1.

6 papers.

  1. 2024Ant Colony Sampling with GFlowNets for Combinatorial Optimizationcited by 53
  2. 2024Genetic-guided GFlowNets for Sample Efficient Molecular Optimizationcited by 36
  3. 2024PolyNet: Learning Diverse Solution Strategies for Neural Combinatorial Optimizationcited by 35
  4. 2025BOPO: Neural Combinatorial Optimization via Best-anchored and Objective-guided Preference Optimizationcited by 14
  5. 2026Learning with Foresight: Enhancing Neural Routing Policy via Multi-Node Lookahead Predictioncited by 1
  6. 2025Preference-Based Gradient Estimation for ML-Guided Approximate Combinatorial Optimizationcited by 0