Efficient Embedding of Semantic Similarity in Control Policies via Entangled Bisimulation
Not on the map. Why?
Abstract (arXiv)
Learning generalizeable policies from visual input in the presence of visual distractions is a challenging problem in reinforcement learning. Recently, there has been renewed interest in bisimulation metrics as a tool to address this issue; these metrics can be used to learn representations that are, in principle, invariant to irrelevant distractions by measuring behavioural similarity between states. An accurate, unbiased, and scalable estimation of these metrics has proved elusive in continuous state and action scenarios. We propose entangled bisimulation, a bisimulation metric that allows the specification of the distance function between states, and can be estimated without bias in continuous state and action spaces. We show how entangled bisimulation can meaningfully improve over previous methods on the Distracting Control Suite (DCS), even when added on top of data augmentation techniques.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 31 references that are in the map. First posted on arXiv 28 Jan 2022.
Builds on
22 of 31 references.
References · 22
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
- 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
- 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
- 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
- 2017Deep Reinforcement Learning that Matterscited by 727
- 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
- 2019Model-Based Reinforcement Learning for Ataricited by 495
- 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
- 2020Reinforcement Learning with Augmented Datacited by 452
10 more
- 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
- 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
- 2020Planning to Explore via Self-Supervised World Modelscited by 365
- 2018Quantifying Generalization in Reinforcement Learningcited by 345
- 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
- 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
- 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 97
- 2019Observational Overfitting in Reinforcement Learningcited by 90
- 2021Towards Robust Bisimulation Metric Learningcited by 49
- 2018Learning Invariances for Policy Generalizationcited by 7
Built on by
Citing papers per year: 2023: 2; 2024: 1.
3 papers.