# Watch Less, Feel More: Sim-to-Real RL for Generalizable Articulated Object Manipulation via Motion Adaptation and Impedance Control

arXiv: https://arxiv.org/abs/2502.14457 · HTML: https://mlsanity.com/paper/2502.14457/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 34 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Builds on (most cited first, 30 of 32)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Affordances from Human Videos as a Versatile Representation for Robotics](https://mlsanity.com/paper/2304.08488/) (2023)
- [Where2Act: From Pixels to Actions for Articulated 3D Objects](https://mlsanity.com/paper/2101.02692/) (2021)
- [ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation](https://mlsanity.com/paper/2312.16217/) (2023)
- [GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Parts](https://mlsanity.com/paper/2211.05272/) (2022)
- [Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion](https://mlsanity.com/paper/2210.10044/) (2022)
- [FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objects](https://mlsanity.com/paper/2205.04382/) (2022)
- [VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects](https://mlsanity.com/paper/2106.14440/) (2021)
- [FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learning](https://mlsanity.com/paper/2401.08553/) (2024)
- [UMI on Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllers](https://mlsanity.com/paper/2407.10353/) (2024)
- [Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion](https://mlsanity.com/paper/2401.17583/) (2024)
- [Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation](https://mlsanity.com/paper/2401.07487/) (2024)
- [UMPNet: Universal Manipulation Policy Network for Articulated Objects](https://mlsanity.com/paper/2109.05668/) (2021)
- [End-to-End Affordance Learning for Robotic Manipulation](https://mlsanity.com/paper/2209.12941/) (2022)
- [Factory: Fast Contact for Robotic Assembly](https://mlsanity.com/paper/2205.03532/) (2022)
- [IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Reality](https://mlsanity.com/paper/2305.17110/) (2023)
- [Graspness Discovery in Clutters for Fast and Accurate Grasp Detection](https://mlsanity.com/paper/2406.11142/) (2024)
- [PartManip: Learning Cross-Category Generalizable Part Manipulation Policy from Point Cloud Observations](https://mlsanity.com/paper/2303.16958/) (2023)
- [Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objects](https://mlsanity.com/paper/2309.07473/) (2023)
- [Adaptive Mobile Manipulation for Articulated Objects In the Open World](https://mlsanity.com/paper/2401.14403/) (2024)
- [Leveraging Language for Accelerated Learning of Tool Manipulation](https://mlsanity.com/paper/2206.13074/) (2022)
- [GAMMA: Graspability-Aware Mobile MAnipulation Policy Learning based on Online Grasping Pose Fusion](https://mlsanity.com/paper/2309.15459/) (2023)
- [Efficient Sim-to-real Transfer of Contact-Rich Manipulation Skills with Online Admittance Residual Learning](https://mlsanity.com/paper/2310.10509/) (2023)
- [Learning Force Control for Legged Manipulation](https://mlsanity.com/paper/2405.01402/) (2024)
- [UniDoorManip: Learning Universal Door Manipulation Policy Over Large-scale and Diverse Door Manipulation Environments](https://mlsanity.com/paper/2403.02604/) (2024)
- [RGBManip: Monocular Image-based Robotic Manipulation through Active Object Pose Estimation](https://mlsanity.com/paper/2310.03478/) (2023)
- [Articulated Object Manipulation with Coarse-to-fine Affordance for Mitigating the Effect of Point Cloud Noise](https://mlsanity.com/paper/2402.18699/) (2024)
- [Learning Variable Impedance Control via Inverse Reinforcement Learning for Force-Related Tasks](https://mlsanity.com/paper/2102.06838/) (2021)
- [Bridging the Sim-to-Real Gap with Dynamic Compliance Tuning for Industrial Insertion](https://mlsanity.com/paper/2311.07499/) (2023)

## Abstract (arXiv)

Articulated object manipulation poses a unique challenge compared to rigid object manipulation as the object itself represents a dynamic environment. In this work, we present a novel RL-based pipeline equipped with variable impedance control and motion adaptation leveraging observation history for generalizable articulated object manipulation, focusing on smooth and dexterous motion during zero-shot sim-to-real transfer. To mitigate the sim-to-real gap, our pipeline diminishes reliance on vision by not leveraging the vision data feature (RGBD/pointcloud) directly as policy input but rather extracting useful low-dimensional data first via off-the-shelf modules. Additionally, we experience less sim-to-real gap by inferring object motion and its intrinsic properties via observation history as well as utilizing impedance control both in the simulation and in the real world. Furthermore, we develop a well-designed training setting with great randomization and a specialized reward system (task-aware and motion-aware) that enables multi-staged, end-to-end manipulation without heuristic motion planning. To the best of our knowledge, our policy is the first to report 84% success rate in the real world via extensive experiments with various unseen objects.