mlsanityData to 14 Oct 2026

Information-driven Affordance Discovery for Efficient Robotic Manipulation

arXiv 2405.03865 ↗First posted 6 May 2024cs.RO cs.AI cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Robotic affordances, providing information about what actions can be taken in a given situation, can aid robotic manipulation. However, learning about affordances requires expensive large annotated datasets of interactions or demonstrations. In this work, we argue that well-directed interactions with the environment can mitigate this problem and propose an information-based measure to augment the agent's objective and accelerate the affordance discovery process. We provide a theoretical justification of our approach and we empirically validate the approach both in simulation and real-world tasks. Our method, which we dub IDA, enables the efficient discovery of visual affordances for several action primitives, such as grasping, stacking objects, or opening drawers, strongly improving data efficiency in simulation, and it allows us to learn grasping affordances in a small number of interactions, on a real-world setup with a UFACTORY XArm 6 robot arm.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 43 of its 55 references that are in the map. First posted on arXiv 6 May 2024.

Builds on

43 of 55 references.

References · 43

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  7. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  8. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  9. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  10. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  11. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  12. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 569
31 more
  1. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  2. 2016Deep Exploration via Bootstrapped DQNcited by 527
  3. 2017A Tutorial on Thompson Samplingcited by 433
  4. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  5. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 344
  6. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
  7. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 307
  8. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 279
  9. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  10. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 224
  11. 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 218
  12. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 182
  13. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  14. 2018Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learningcited by 128
  15. 2022Deep Hierarchical Planning from Pixelscited by 101
  16. 2020Learning Dexterous Grasping with Object-Centric Visual Affordancescited by 85
  17. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  18. 2022Pessimistic Q-Learning for Offline Reinforcement Learning: Towards Optimal Sample Complexitycited by 85
  19. 2017Boltzmann Exploration Done Rightcited by 69
  20. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 65
  21. 2022Packed-Ensembles for Efficient Uncertainty Estimationcited by 49
  22. 2021O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learningcited by 46
  23. 2021AW-Opt: Learning Robotic Skills with Imitation and Reinforcement at Scalecited by 44
  24. 2022Affordance Learning from Play for Sample-Efficient Policy Learningcited by 30
  25. 2021What Can I Do Here? Learning New Skills by Imagining Visual Affordancescited by 23
  26. 2020Multi-Fingered Active Grasp Learningcited by 13
  27. 2022PLATO: Predicting Latent Affordances Through Object-Centric Playcited by 13
  28. 2020Exploratory Grasping: Asymptotically Optimal Algorithms for Grasping Challenging Polyhedral Objectscited by 9
  29. 2022Grasp Learning: Models, Methods, and Performancecited by 8
  30. 2021LEGS: Learning Efficient Grasp Sets for Exploratory Graspingcited by 5
  31. 2020Accelerating Grasp Exploration by Leveraging Learned Priorscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 35
  2. 2024Redundancy-aware Action Spaces for Robot Learningcited by 5
  3. 2025ASkDAgger: Active Skill-level Data Aggregation for Interactive Imitation Learningcited by 0
  4. 2026What Objects Enable, Not What They Are: Functional Latent Spaces for Affordance Reasoningcited by 0