Quality-Diversity with Limited Resources
Not on the map. Why?
Abstract (arXiv)
Quality-Diversity (QD) algorithms have emerged as a powerful optimization paradigm with the aim of generating a set of high-quality and diverse solutions. To achieve such a challenging goal, QD algorithms require maintaining a large archive and a large population in each iteration, which brings two main issues, sample and resource efficiency. Most advanced QD algorithms focus on improving the sample efficiency, while the resource efficiency is overlooked to some extent. Particularly, the resource overhead during the training process has not been touched yet, hindering the wider application of QD algorithms. In this paper, we highlight this important research question, i.e., how to efficiently train QD algorithms with limited resources, and propose a novel and effective method called RefQD to address it. RefQD decomposes a neural network into representation and decision parts, and shares the representation part with all decision parts in the archive to reduce the resource overhead. It also employs a series of strategies to address the mismatch issue between the old decision parts and the newly updated representation part. Experiments on different types of tasks from small to large resource consumption demonstrate the excellent performance of RefQD: it not only uses significantly fewer resources (e.g., 16% GPU memories on QDax and 3.7% on Atari) but also achieves comparable or better performance compared to sample-efficient QD algorithms. Our code is available at urlhttps://github.com/lamda-bbo/RefQD.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 39 of its 57 references that are in the map. First posted on arXiv 6 Jun 2024.
Builds on
39 of 57 references.
References · 39
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
- 2015Illuminating search spaces by mapping elitescited by 425
- 2020First return, then explorecited by 266
- 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 205
- 2014Robots that can adapt like animalscited by 204
- 2021Collaborating with Humans without Human Datacited by 142
- 2017Quality and Diversity Optimization: A Unifying Modular Frameworkcited by 113
- 2021RRL: Resnet as representation for Reinforcement Learningcited by 91
- 2020One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RLcited by 79
27 more
- 2019Covariance Matrix Adaptation for the Rapid Illumination of Behavior Spacecited by 78
- 2021Differentiable Quality Diversitycited by 72
- 2020The Value-Improvement Path: Towards Better Representations for Reinforcement Learningcited by 60
- 2020Quality-Diversity Optimization: a novel branch of stochastic optimizationcited by 60
- 2020Diversity Policy Gradient for Sample Efficient Quality-Diversity Optimizationcited by 54
- 2020Illuminating Mario Scenes in the Latent Space of a Generative Adversarial Networkcited by 52
- 2018Discovering the Elite Hypervolume by Leveraging Interspecies Correlationcited by 41
- 2022Deep Surrogate Assisted Generation of Environmentscited by 40
- 2020Scaling MAP-Elites to Deep Neuroevolutioncited by 38
- 2022Approximating Gradients for Differentiable Quality Diversity in Reinforcement Learningcited by 38
- 2021Unsupervised Behaviour Discovery with Quality-Diversity Optimisationcited by 30
- 2022Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discoverycited by 23
- 2023MAP-Elites with Descriptor-Conditioned Gradients and Archive Distillation into a Single Policycited by 20
- 2023Generating Behaviorally Diverse Policies with Latent Diffusion Modelscited by 19
- 2023Multi-Robot Coordination and Layout Design for Automated Warehousingcited by 17
- 2023QDax: A Library for Quality-Diversity and Population-based Algorithms with Hardware Accelerationcited by 17
- 2022Empirical analysis of PGA-MAP-Elites for Neuroevolution in Uncertain Domainscited by 15
- 2022Accelerated Quality-Diversity through Massive Parallelismcited by 14
- 2022ERL-Re2: Efficient Evolutionary Reinforcement Learning with Shared State Representation and Individual Policy Representationcited by 14
- 2020Fast and stable MAP-Elites in noisy domains using deep gridscited by 13
- 2021Monte Carlo Elites: Quality-Diversity Selection as a Multi-Armed Bandit Problemcited by 10
- 2023The Quality-Diversity Transformer: Generating Behavior-Conditioned Trajectories with Decision Transformerscited by 10
- 2023Quality Diversity through Human Feedback: Towards Open-Ended Diversity-Driven Optimizationcited by 10
- 2024Quality-Diversity Algorithms Can Provably Be Helpful for Optimizationcited by 9
- 2021Few-shot Quality-Diversity Optimizationcited by 8
- 2023Diversity from Human Feedbackcited by 5
- 2023Don't Bet on Luck Alone: Enhancing Behavioral Reproducibility of Quality-Diversity Solutions in Uncertain Domainscited by 3
Built on by
Citing papers per year: 2025: 1.
1 paper.