mlsanityData to 7 Oct 2026

Neuro-algorithmic Policies enable Fast Combinatorial Generalization

arXiv 2102.07456 ↗First posted 15 Feb 2021cs.LG cs.AI cs.DMcited by 11

Not on the map. Why?

Abstract (arXiv)

Although model-based and model-free approaches to learning the control of systems have achieved impressive results on standard benchmarks, generalization to task variations is still lacking. Recent results suggest that generalization for standard architectures improves only after obtaining exhaustive amounts of data. We give evidence that generalization capabilities are in many cases bottlenecked by the inability to generalize on the combinatorial aspects of the problem. Furthermore, we show that for a certain subclass of the MDP framework, this can be alleviated by neuro-algorithmic architectures. Many control problems require long-term planning that is hard to solve generically with neural networks alone. We introduce a neuro-algorithmic policy architecture consisting of a neural network and an embedded time-dependent shortest path solver. These policies can be trained end-to-end by blackbox differentiation. We show that this type of architecture generalizes well to unseen variations in the environment already after seeing a few examples.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 40 of its 61 references that are in the map. First posted on arXiv 15 Feb 2021.

Builds on

40 of 61 references.

References · 40

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  4. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  5. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  6. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  7. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  8. 2019Model-Based Reinforcement Learning for Ataricited by 495
  9. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  10. 2020Reinforcement Learning with Augmented Datacited by 452
  11. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  12. 2016Input Convex Neural Networkscited by 407
28 more
  1. 2017Smart "Predict, then Optimize"cited by 332
  2. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  3. 2019Differentiation of Blackbox Combinatorial Solverscited by 224
  4. 2018Semi-parametric Topological Memory for Navigationcited by 203
  5. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  6. 2019SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solvercited by 173
  7. 2016Value Iteration Networkscited by 158
  8. 2017Imagination-Augmented Agents for Deep Reinforcement Learningcited by 137
  9. 2017Value Prediction Networkcited by 117
  10. 2016Model-Free Episodic Controlcited by 100
  11. 2019MIPaaL: Mixed Integer Program as a Layercited by 93
  12. 2016The Predictron: End-To-End Learning and Planningcited by 92
  13. 2019Smart Predict-and-Optimize for Hard Combinatorial Optimization Problemscited by 88
  14. 2018Optimal Control Via Neural Networks: A Convex Approachcited by 82
  15. 2020Learning with Differentiable Perturbed Optimizerscited by 81
  16. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  17. 2019Learning Compositional Koopman Operators for Model-Based Controlcited by 67
  18. 2018SparseMAP: Differentiable Sparse Structured Inferencecited by 62
  19. 2017Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundarycited by 58
  20. 2019Optimizing Rank-based Metrics with Blackbox Differentiationcited by 58
  21. 2020Path Planning using Neural A* Searchcited by 51
  22. 2014Informed RRT*: Optimal Sampling-based Path Planning Focused via Direct Sampling of an Admissible Ellipsoidal Heuristiccited by 50
  23. 2020Model-Predictive Control via Cross-Entropy and Gradient-Based Optimizationcited by 32
  24. 2019The Differentiable Cross-Entropy Methodcited by 26
  25. 2019Differentiable Algorithm Networks for Composable Robot Learningcited by 24
  26. 2019LEGO: Leveraging Experience in Roadmap Generation for Sampling-Based Planningcited by 19
  27. 2018Deep sequential models for sampling-based planningcited by 7
  28. 2019Towards Stable Symbol Grounding with Zero-Suppressed State AutoEncodercited by 5

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 3; 2023: 3; 2024: 2; 2025: 1.

11 papers.

  1. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  2. 2022Backpropagation through Combinatorial Algorithms: Identity with Projection Workscited by 29
  3. 2023Explore to Generalize in Zero-Shot RLcited by 20
  4. 2024Causal Action Influence Aware Counterfactual Data Augmentationcited by 12
  5. 2023Optimize Planning Heuristics to Rank, not to Estimate Cost-to-Goalcited by 10
  6. 2023Towards Combinatorial Generalization for Catalysts: A Kohn-Sham Charge-Density Approachcited by 5
  7. 2021Planning from Pixels in Environments with Combinatorially Hard Search Spacescited by 4
  8. 2022A Differentiable Loss Function for Learning Heuristics in A*cited by 2
  9. 2022Evolving Pareto-Optimal Actor-Critic Algorithms for Generalizability and Stabilitycited by 1
  10. 2025A Novel Switch-Type Policy Network for Resource Allocation Problems: Technical Reportcited by 1
  11. 2024SLOPE: Search with Learned Optimal Pruning-based Expansioncited by 0