mlsanityData to 7 Oct 2026

One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL

arXiv 2010.14484 ↗First posted 27 Oct 2020cs.LGcited by 79

Not on the map. Why?

Abstract (arXiv)

While reinforcement learning algorithms can learn effective policies for complex tasks, these policies are often brittle to even minor task variations, especially when variations are not explicitly provided during training. One natural approach to this problem is to train agents with manually specified variation in the training task or environment. However, this may be infeasible in practical situations, either because making perturbations is not possible, or because it is unclear how to choose suitable perturbation strategies without sacrificing performance. The key insight of this work is that learning diverse behaviors for accomplishing a task can directly lead to behavior that generalizes to varying environments, without needing to perform explicit perturbations during training. By identifying multiple solutions for the task in a single environment during training, our approach can generalize to new situations by abandoning solutions that are no longer effective and adopting those that are. We theoretically characterize a robustness set of environments that arises from our algorithm and empirically find that our diversity-driven approach can extrapolate to various changes in the environment and task.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 19; 2023: 21; 2024: 12; 2025: 14; 2026: 2.

79 papers.

Sort:

By area:

  1. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  2. 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 113
  3. 2022Diversify and Disambiguate: Learning From Underspecified Datacited by 55
  4. 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrationscited by 51
  5. 2022DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learningcited by 40
  6. 2021Learning more skills through optimistic explorationcited by 37
  7. 2022Approximating Gradients for Differentiable Quality Diversity in Reinforcement Learningcited by 37
  8. 2022Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimalitycited by 35
  9. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  10. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
  11. 2023The Generalization Gap in Offline Reinforcement Learningcited by 25
  12. 2021Specializing Versatile Skill Libraries using Local Mixture of Expertscited by 24
38 more
  1. 2022Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guaranteescited by 24
  2. 2022Continuously Discovering Novel Strategies via Reward-Switching Policy Optimizationcited by 23
  3. 2022Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discoverycited by 23
  4. 2022Leveraging Approximate Symbolic Models for Reinforcement Learning via Skill Diversitycited by 22
  5. 2021Discovering Diverse Nearly Optimal Policies with Successor Featurescited by 20
  6. 2021Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoirescited by 20
  7. 2023Reparameterized Policy Learning for Multimodal Trajectory Optimizationcited by 20
  8. 2023ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPscited by 19
  9. 2021Discovering Diverse Solutions in Deep Reinforcement Learning by Maximizing State-Action-Based Mutual Informationcited by 18
  10. 2024Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RLcited by 17
  11. 2022Robust Policy Learning over Multiple Uncertainty Setscited by 16
  12. 2023QDax: A Library for Quality-Diversity and Population-based Algorithms with Hardware Accelerationcited by 16
  13. 2023Pre-Training and Fine-Tuning Generative Flow Networkscited by 16
  14. 2023Policy Space Diversity for Non-Transitive Gamescited by 15
  15. 2023Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RLcited by 15
  16. 2023Diversifying AI: Towards Creative Chess with AlphaZerocited by 15
  17. 2021Learning a subspace of policies for online adaptation in Reinforcement Learningcited by 14
  18. 2021Motion Planning by Learning the Solution Manifold in Trajectory Optimizationcited by 13
  19. 2022Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motionscited by 13
  20. 2024Quality-Diversity Actor-Critic: Learning High-Performing and Diverse Behaviors via Value and Successor Features Criticscited by 13
  21. 2021Adaptable Agent Populations via a Generative Model of Policiescited by 12
  22. 2022Heterogeneous Multi-agent Zero-Shot Coordination by Coevolutioncited by 12
  23. 2025Evolution and The Knightian Blindspot of Machine Learningcited by 11
  24. 2022DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimizationcited by 10
  25. 2024Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policiescited by 8
  26. 2024Do's and Don'ts: Learning Desirable Skills with Instruction Videoscited by 8
  27. 2024Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNetscited by 8
  28. 2025The impact of intrinsic rewards on exploration in Reinforcement Learningcited by 8
  29. 2025Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Modelscited by 8
  30. 2023Learning Diverse Skills for Local Navigation under Multi-constraint Optimalitycited by 7
  31. 2025Polychromic Objectives for Reinforcement Learningcited by 7
  32. 2021Learning Meta Representations for Agents in Multi-Agent Reinforcement Learningcited by 6
  33. 2023Synergizing Quality-Diversity with Descriptor-Conditioned Reinforcement Learningcited by 6
  34. 2023Reinforcement Learning by Guided Safe Explorationcited by 5
  35. 2023Keep Various Trajectories: Promoting Exploration of Ensemble Policies in Continuous Controlcited by 5
  36. 2024DIDI: Diffusion-Guided Diversity for Offline Behavioral Generationcited by 4
  37. 2024Discovering Multiple Solutions from a Single Task in Offline Reinforcement Learningcited by 4
  38. 2023Efficient Quality-Diversity Optimization through Diverse Quality Speciescited by 3

50 most cited of 79 All 79 (JSON)