Build generally reusable agent-environment interaction models
Not on the map. Why?
Abstract (arXiv)
This paper tackles the problem of how to pre-train a model and make it generally reusable backbones for downstream task learning. In pre-training, we propose a method that builds an agent-environment interaction model by learning domain invariant successor features from the agent's vast experiences covering various tasks, then discretize them into behavior prototypes which result in an embodied set structure. To make the model generally reusable for downstream task learning, we propose (1) embodied feature projection that retains previous knowledge by projecting the new task's observation-action pair to the embodied set structure and (2) projected Bellman updates which add learning plasticity for the new task setting. We provide preliminary results that show downstream task learning based on a pre-trained embodied set structure can handle unseen changes in task objectives, environmental dynamics and sensor modalities.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 31 of its 42 references that are in the map. First posted on arXiv 13 Nov 2022.
Builds on
31 of 42 references.
References · 31
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
- 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
- 2018World Modelscited by 1,444
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
- 2018Experience Replay for Continual Learningcited by 1,074
- 2022A Generalist Agentcited by 802
- 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
19 more
- 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
- 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
- 2020Reinforcement Learning with Augmented Datacited by 452
- 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
- 2016Successor Features for Transfer in Reinforcement Learningcited by 331
- 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
- 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
- 2016Variational Intrinsic Controlcited by 262
- 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 136
- 2018Assessing Generalization in Deep Reinforcement Learningcited by 132
- 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
- 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
- 2018Deep Variational Reinforcement Learning for POMDPscited by 122
- 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 114
- 2021RRL: Resnet as representation for Reinforcement Learningcited by 89
- 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
- 2021Successor Feature Landmarks for Long-Horizon Goal-Conditioned Reinforcement Learningcited by 23
- 2019On Value Functions and the Agent-Environment Boundarycited by 19
- 2020Offline Learning of Counterfactual Predictions for Real-World Robotic Reinforcement Learningcited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.