Tackling Visual Control via Multi-View Exploration Maximization
Not on the map. Why?
Abstract (arXiv)
We present MEM: Multi-view Exploration Maximization for tackling complex visual control tasks. To the best of our knowledge, MEM is the first approach that combines multi-view representation learning and intrinsic reward-driven exploration in reinforcement learning (RL). More specifically, MEM first extracts the specific and shared information of multi-view observations to form high-quality features before performing RL on the learned features, enabling the agent to fully comprehend the environment and yield better actions. Furthermore, MEM transforms the multi-view features into intrinsic rewards based on entropy maximization to encourage exploration. As a result, MEM can significantly promote the sample-efficiency and generalization ability of the RL agent, facilitating solving real-world problems with high-dimensional observations and spare-reward space. We evaluate MEM on various tasks from DeepMind Control Suite and Procgen games. Extensive simulation results demonstrate that MEM can achieve superior performance and outperform the benchmarking schemes with simple architecture and higher efficiency.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 36 of its 59 references that are in the map. First posted on arXiv 28 Nov 2022.
Builds on
36 of 59 references.
References · 36
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2018Exploration by Random Network Distillationcited by 966
- 2018DeepMind Control Suitecited by 935
- 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
- 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
24 more
- 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
- 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
- 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
- 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
- 2020Reinforcement Learning with Augmented Datacited by 452
- 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
- 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
- 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
- 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
- 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
- 2017Count-Based Exploration with Neural Density Modelscited by 255
- 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
- 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
- 2020Learning Robust Representations via Multi-View Information Bottleneckcited by 189
- 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
- 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
- 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
- 2020Deep Reinforcement and InfoMax Learningcited by 79
- 2020Intrinsic Reward Driven Imitation Learning via Generative Modelcited by 35
- 2021DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneckcited by 31
- 2018The Utility of Sparse Representations for Control in Reinforcement Learningcited by 23
- 2022R'enyi State Entropy for Exploration Acceleration in Reinforcement Learningcited by 18
- 2019Multi-View Reinforcement Learningcited by 12
- 2022Rewarding Episodic Visitation Discrepancy for Exploration in Reinforcement Learningcited by 7
Built on by
Citing papers per year: 2023: 1.
1 paper.