Data to 7 Oct 2026

CoBERL: Contrastive BERT for Reinforcement Learning

arXiv 2107.05431 ↗First posted 12 Jul 2021cs.LG cs.AIcited by 20

Not on the map. Why?

Abstract (arXiv)

Many reinforcement learning (RL) agents require a large amount of experience to solve tasks. We propose Contrastive BERT for RL (CoBERL), an agent that combines a new contrastive loss and a hybrid LSTM-transformer architecture to tackle the challenge of improving data efficiency. CoBERL enables efficient, robust learning from pixels across a wide range of domains. We use bidirectional masked prediction in combination with a generalization of recent contrastive methods to learn better representations for transformers in RL, without the need of hand engineered data augmentations. We find that CoBERL consistently improves performance across the full Atari suite, a set of control tasks and a challenging 3D environment.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 38 of its 57 references that are in the map. First posted on arXiv 12 Jul 2021.

Builds on

38 of 57 references.

References · 38

2017Attention Is All You Needcited by 59,932
2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
2020Language Models are Few-Shot Learnerscited by 32,199
2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
2018Representation Learning with Contrastive Predictive Codingcited by 7,662
2020End-to-End Object Detection with Transformerscited by 7,384
2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
2013Playing Atari with Deep Reinforcement Learningcited by 3,211
2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830

26 more

2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
2018World Modelscited by 1,444
2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
2020Mastering Atari with Discrete World Modelscited by 990
2018DeepMind Control Suitecited by 935
2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
2019Model-Based Reinforcement Learning for Ataricited by 495
2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
2018An Empirical Model of Large-Batch Trainingcited by 268
2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
2019Stabilizing Transformers for Reinforcement Learningcited by 262
2020Representation Learning via Invariant Causal Mechanismscited by 197
2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
2019V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Controlcited by 85
2016Learning values across many orders of magnitudecited by 80
2020Deep Reinforcement and InfoMax Learningcited by 79
2019Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languagescited by 37
2021Podracer architectures for scalable Reinforcement Learningcited by 37
2021Return-Based Contrastive Representation Learning for Reinforcement Learningcited by 36
2019Generalization of Reinforcement Learners with Working and Episodic Memorycited by 35
2018Relative Entropy Regularized Policy Iterationcited by 32
2020Masked Contrastive Representation Learning for Reinforcement Learningcited by 32
2021The Uncanny Similarity of Recurrence and Depthcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 6; 2024: 4; 2025: 1; 2026: 2.

20 papers.

Sort:

By area:

2022Retrieval-Augmented Reinforcement Learningcited by 53
2023A Survey on Transformers in Reinforcement Learningcited by 50
2023SMART: Self-supervised Multi-task pretrAining with contRol Transformerscited by 44
2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
2021Procedural Generalization by Planning with Self-Supervised World Modelscited by 30
2022Abstraction for Deep Reinforcement Learningcited by 19
2022Light-weight probing of unsupervised representations for Reinforcement Learningcited by 15
2023Transformers in Reinforcement Learning: A Surveycited by 11
2024CORN: Contact-based Object Representation for Nonprehensile Manipulation of General Unseen Objectscited by 8
2022Transformer in Transformer as Backbone for Deep Reinforcement Learningcited by 6
2023RePreM: Representation Pre-training with Masked Model for Reinforcement Learningcited by 5
2023PASTA: Pretrained Action-State Transformer Agentscited by 5

8 more

2024Learning Goal-Conditioned Representations for Language Reward Modelscited by 2
2024Contrastive Abstraction for Reinforcement Learningcited by 2
2024MarineFormer: A Spatio-Temporal Attention Model for USV Navigation in Dynamic Marine Environmentscited by 1
2025Stackelberg Coupling of Online Representation Learning and Reinforcement Learningcited by 1
2022Using Forwards-Backwards Models to Approximate MDP Homomorphismscited by 0
2023Differentially Private Optimization for Non-Decomposable Objective Functionscited by 0
2026Transformer-Enhanced Reinforcement Learning: Fundamentals and Applications in Communication Networkscited by 0
2026Statistical Convergence of Transformer Encoder-Accelerated Robust Reinforcement Learningcited by 0