Data to 7 Oct 2026

Loss is its own Reward: Self-Supervision for Reinforcement Learning

arXiv 1612.07307 ↗First posted 21 Dec 2016cs.LGcited by 72

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning optimizes policies for expected cumulative reward. Need the supervision be so narrow? Reward is delayed and sparse for many tasks, making it a difficult and impoverished signal for end-to-end optimization. To augment reward, we consider a range of self-supervised tasks that incorporate states, actions, and successors to provide auxiliary losses. These losses offer ubiquitous and instantaneous supervision for representation learning even in the absence of reward. While current results show that learning from reward alone is feasible, pure reinforcement learning methods are constrained by computational and data efficiency issues that can be remedied by auxiliary losses. Self-supervised pre-training and joint optimization improve the data efficiency and policy returns of end-to-end reinforcement learning.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 23; 2022: 18; 2023: 13; 2024: 8; 2025: 2; 2026: 6.

72 papers.

Sort:

By area:

2022Masked Visual Pre-training for Motor Controlcited by 246
2022Masked World Models for Visual Controlcited by 169
2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 90
2022Audio Self-supervised Learning: A Surveycited by 65
2021Investigating the Role of Negatives in Contrastive Representation Learningcited by 54
2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 52
2024DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Controlcited by 43
2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 42
2021PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learningcited by 41
2022AdaMAE: Adaptive Masking for Efficient Spatiotemporal Learning with Masked Autoencoderscited by 41

38 more

2022Mask-based Latent Reconstruction for Reinforcement Learningcited by 39
2021Training Larger Networks for Deep Reinforcement Learningcited by 37
2021Return-Based Contrastive Representation Learning for Reinforcement Learningcited by 36
2021Which Mutual-Information Representation Learning Objectives are Sufficient for Control?cited by 28
2022INFOrmation Prioritization through EmPOWERment in Visual Model-Based RLcited by 27
2021Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoningcited by 25
2022Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?cited by 24
2022Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machinescited by 20
2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learningcited by 15
2021Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Explorationcited by 14
2021SimSR: Simple Distance-based State Representation for Deep Reinforcement Learningcited by 14
2022Reinforcement Learning with Automated Auxiliary Loss Searchcited by 12
2022Behavior Prior Representation learning for Offline Reinforcement Learningcited by 12
2021Learning Temporally-Consistent Representations for Data-Efficient Reinforcement Learningcited by 10
2022Learning Robust Real-Time Cultural Transmission without Human Datacited by 9
2022MLGOPerf: An ML Guided Inliner to Optimize Performancecited by 9
2021Improving Context-Based Meta-Reinforcement Learning with Self-Supervised Trajectory Contrastive Learningcited by 8
2021Multitasking Inhibits Semantic Driftcited by 8
2024PREDILECT: Preferences Delineated with Zero-Shot Language-based Reasoning in Reinforcement Learningcited by 8
2026What Makes Video World Model Latents Action-Relevant: Prediction over Reconstructioncited by 8
2021Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learningcited by 7
2023Predictive auxiliary objectives in deep RL mimic learning in the braincited by 7
2022Transfer Dynamics in Emergent Evolutionary Curriculacited by 6
2019State Representation Learning from Demonstrationcited by 5
2021Lifelong Hyper-Policy Optimization with Multiple Importance Sampling Regularizationcited by 5
2022What makes useful auxiliary tasks in reinforcement learning: investigating the effect of the target policycited by 5
2023RePreM: Representation Pre-training with Masked Model for Reinforcement Learningcited by 5
2022Sampling Through the Lens of Sequential Decision Makingcited by 4
2023RLAD: Reinforcement Learning from Pixels for Autonomous Driving in Urban Environmentscited by 4
2023Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Secondcited by 4
2026No Gaussian Required: Contrastive Inverse Dynamics for JEPA World Modelscited by 4
2021A Study on Dense and Sparse (Visual) Rewards in Robot Policy Learningcited by 3
2021DiffSRL: Learning Dynamical State Representation for Deformable Object Manipulation with Differentiable Simulatorcited by 3
2022Learning List-wise Representation in Reinforcement Learning for Ads Allocation with Multiple Auxiliary Taskscited by 3
2023Decentralized Multi-Agent Reinforcement Learning with Global State Predictioncited by 3
2023Policy Contrastive Imitation Learningcited by 3
2023A Kernel Perspective on Behavioural Metrics for Markov Decision Processescited by 3
2024MOOSS: Mask-Enhanced Temporal Contrastive Learning for Smooth State Evolution in Visual Reinforcement Learningcited by 3

50 most cited of 72 All 72 (JSON)