mlsanityData to 7 Oct 2026

What Can I Do Here? Learning New Skills by Imagining Visual Affordances

arXiv 2106.00671 ↗First posted 1 Jun 2021cs.RO cs.CV cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

A generalist robot equipped with learned skills must be able to perform many tasks in many different environments. However, zero-shot generalization to new settings is not always possible. When the robot encounters a new environment or object, it may need to finetune some of its previously learned skills to accommodate this change. But crucially, previously learned behaviors and models should still be suitable to accelerate this relearning. In this paper, we aim to study how generative models of possible outcomes can allow a robot to learn visual representations of affordances, so that the robot can sample potentially possible outcomes in new situations, and then further train its policy to achieve those outcomes. In effect, prior data is used to learn what kinds of outcomes may be possible, such that when the robot encounters an unfamiliar setting, it can sample potential outcomes from its model, attempt to reach them, and thereby update both its skills and its outcome model. This approach, visuomotor affordance learning (VAL), can be used to train goal-conditioned policies that operate on raw image inputs, and can rapidly learn to manipulate new objects via our proposed affordance-directed exploration scheme. We show that VAL can utilize prior data to solve real-world tasks such drawer opening, grasping, and placing objects in new scenes with only five minutes of online experience in the new scene.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 29 of its 55 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

29 of 55 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2017Neural Discrete Representation Learningcited by 4,650
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  7. 2017Hindsight Experience Replaycited by 1,096
  8. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  9. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  10. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  11. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  12. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
17 more
  1. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  2. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  3. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  4. 2016Adversarial Feature Learningcited by 350
  5. 2019Learning Latent Plans from Playcited by 310
  6. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  7. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  8. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  9. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  10. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  11. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
  12. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  13. 2013Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robotscited by 56
  14. 2020Deep Affordance Foresight: Planning Through What Can Be Done in the Futurecited by 41
  15. 2019Contextual Imagined Goals for Self-Supervised Robotic Learningcited by 40
  16. 2020What can I do here? A Theory of Affordances in Reinforcement Learningcited by 33
  17. 2018Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Explorationcited by 23

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 8; 2023: 5; 2024: 3; 2025: 3.

22 papers.

Sort:

By area:

  1. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 234
  2. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  3. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 126
  4. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 68
  5. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 52
  6. 2022Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learningcited by 38
  7. 2022Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Spacecited by 31
  8. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
  9. 2022Generalization with Lossy Affordances: Leveraging Broad Offline Data for Learning Visuomotor Taskscited by 24
  10. 2021Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learningcited by 17
  11. 2023Visual Affordance Prediction for Guiding Robot Explorationcited by 12
  12. 2022Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Visioncited by 11
10 more
  1. 2022Possibility Before Utility: Learning And Using Hierarchical Affordancescited by 4
  2. 2024Information-driven Affordance Discovery for Efficient Robotic Manipulationcited by 4
  3. 2025Null Counterfactual Factor Interactions for Goal-Conditioned Reinforcement Learningcited by 4
  4. 2023Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planningcited by 3
  5. 2022Weakly Supervised Disentangled Representation for Goal-conditioned Reinforcement Learningcited by 2
  6. 2023Learning Sim-to-Real Dense Object Descriptors for Robotic Manipulationcited by 2
  7. 2023Hierarchical Reinforcement Learning in Complex 3D Environmentscited by 0
  8. 2024The Interpretability of Codebooks in Model-Based Reinforcement Learning is Limitedcited by 0
  9. 2025Proposing Hierarchical Goal-Conditioned Policy Planning in Multi-Goal Reinforcement Learningcited by 0
  10. 2025A representational framework for learning and encoding structurally enriched trajectories in complex agent environmentscited by 0