# Learning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objects

arXiv: https://arxiv.org/abs/2209.05802 · HTML: https://mlsanity.com/paper/2209.05802/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2022.

## Built on by (most cited first, 16 of 16)

- [QUAR-VLA: Vision-Language-Action Model for Quadruped Robots](https://mlsanity.com/paper/2312.14457/) (2023)
- [Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objects](https://mlsanity.com/paper/2309.07473/) (2023)
- [FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction](https://mlsanity.com/paper/2503.07909/) (2025)
- [Exploring Embodied Multimodal Large Models: Development, Datasets, and Future Directions](https://mlsanity.com/paper/2502.15336/) (2025)
- [Learning Primitive Embodied World Models: Towards Scalable Robotic Learning](https://mlsanity.com/paper/2508.20840/) (2025)
- [SM3: Self-Supervised Multi-task Modeling with Multi-view 2D Images for Articulated Objects](https://mlsanity.com/paper/2401.09133/) (2024)
- [Online Estimation and Manipulation of Articulated Objects](https://mlsanity.com/paper/2601.01438/) (2026)
- [Multi-Object Graph Affordance Network: Goal-Oriented Planning through Learned Compound Object Affordances](https://mlsanity.com/paper/2309.10426/) (2023)
- [Visual-Geometric Collaborative Guidance for Affordance Learning](https://mlsanity.com/paper/2410.11363/) (2024)
- [Opening Articulated Structures in the Real World](https://mlsanity.com/paper/2402.17767/) (2024)
- [Embodied AI in Machine Learning -- is it Really Embodied?](https://mlsanity.com/paper/2505.10705/) (2025)
- [WestWorld: A Knowledge-Encoded Scalable Trajectory World Model for Diverse Robotic Systems](https://mlsanity.com/paper/2603.14392/) (2026)
- [Revisiting Proprioceptive Sensing for Articulated Object Manipulation](https://mlsanity.com/paper/2305.09584/) (2023)
- [Robotic Test Tube Rearrangement Using Combined Reinforcement Learning and Motion Planning](https://mlsanity.com/paper/2401.09772/) (2024)
- [ArtReg: Visuo-Tactile based Pose Tracking and Manipulation of Unseen Articulated Objects](https://mlsanity.com/paper/2511.06378/) (2025)
- [Scalable Trajectory Generation for Whole-Body Mobile Manipulation](https://mlsanity.com/paper/2604.12565/) (2026)

## Abstract (arXiv)

Interactions with articulated objects are a challenging but important task for mobile robots. To tackle this challenge, we propose a novel closed-loop control pipeline, which integrates manipulation priors from affordance estimation with sampling-based whole-body control. We introduce the concept of agent-aware affordances which fully reflect the agent's capabilities and embodiment and we show that they outperform their state-of-the-art counterparts which are only conditioned on the end-effector geometry. Additionally, closed-loop affordance inference is found to allow the agent to divide a task into multiple non-continuous motions and recover from failure and unexpected states. Finally, the pipeline is able to perform long-horizon mobile manipulation tasks, i.e. opening and closing an oven, in the real world with high success rates (opening: 71%, closing: 72%).