# ActivePusher: Active Learning and Planning with Residual Physics for Nonprehensile Manipulation

arXiv: https://arxiv.org/abs/2506.04646 · HTML: https://mlsanity.com/paper/2506.04646/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 40 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 29 of 29)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [Deep Visual Foresight for Planning Robot Motion](https://mlsanity.com/paper/1610.00696/) (2016)
- [Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images](https://mlsanity.com/paper/1506.07365/) (2015)
- [Learning to Poke by Poking: Experiential Learning of Intuitive Physics](https://mlsanity.com/paper/1606.07419/) (2016)
- [TossingBot: Learning to Throw Arbitrary Objects with Residual Physics](https://mlsanity.com/paper/1903.11239/) (2019)
- [Gone Fishing: Neural Active Learning with Fisher Embeddings](https://mlsanity.com/paper/2106.09675/) (2021)
- [MuJoCo Playground](https://mlsanity.com/paper/2502.08844/) (2025)
- [A Framework and Benchmark for Deep Batch Active Learning for Regression](https://mlsanity.com/paper/2203.09410/) (2022)
- [Learning compositional models of robot skills for task and motion planning](https://mlsanity.com/paper/2006.06444/) (2020)
- [Augmenting Physical Simulators with Stochastic Neural Networks: Case Study of Planar Pushing and Bouncing](https://mlsanity.com/paper/1808.03246/) (2018)
- [HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulation](https://mlsanity.com/paper/2305.03942/) (2023)
- [Active Learning in Robotics: A Review of Control Principles](https://mlsanity.com/paper/2106.13697/) (2021)
- [Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels](https://mlsanity.com/paper/2206.12569/) (2022)
- [Practice Makes Perfect: Planning to Learn Skill Parameter Policies](https://mlsanity.com/paper/2402.15025/) (2024)
- [Sampling-Based Motion Planning: A Comparative Review](https://mlsanity.com/paper/2309.13119/) (2023)
- [Asymptotically Optimal Sampling-based Kinodynamic Planning](https://mlsanity.com/paper/1407.2896/) (2014)
- [A Data-Efficient Approach to Precise and Controlled Pushing](https://mlsanity.com/paper/1807.09904/) (2018)
- [Promises of Deep Kernel Learning for Control Synthesis](https://mlsanity.com/paper/2309.06569/) (2023)
- [UNO Push: Unified Nonprehensile Object Pushing via Non-Parametric Estimation and Model Predictive Control](https://mlsanity.com/paper/2403.13274/) (2024)
- [Learning Manipulation States and Actions for Efficient Non-prehensile Rearrangement Planning](https://mlsanity.com/paper/1901.03557/) (2019)
- [Object-Centric Kinodynamic Planning for Nonprehensile Robot Rearrangement Manipulation](https://mlsanity.com/paper/2410.00261/) (2024)
- [Kinodynamic Rapidly-exploring Random Forest for Rearrangement-Based Nonprehensile Manipulation](https://mlsanity.com/paper/2302.04360/) (2023)
- [Task-Oriented Active Learning of Model Preconditions for Inaccurate Dynamics Models](https://mlsanity.com/paper/2401.04007/) (2024)
- [Online Adaptation of Sampling-Based Motion Planning with Inaccurate Models](https://mlsanity.com/paper/2403.07638/) (2024)
- [Expansion-GRR: Efficient Generation of Smooth Global Redundancy Resolution Roadmaps](https://mlsanity.com/paper/2405.13770/) (2024)

## Abstract (arXiv)

Planning with learned dynamics models offers a promising approach toward versatile real-world manipulation, particularly in nonprehensile settings such as pushing or rolling, where accurate analytical models are difficult to obtain. However, collecting training data for learning-based methods can be costly and inefficient, as it often relies on randomly sampled interactions that are not necessarily the most informative. Furthermore, learned models tend to exhibit high uncertainty in underexplored regions of the skill space, undermining the reliability of long-horizon planning. To address these challenges, we propose ActivePusher, a novel framework that combines residual-physics modeling with uncertainty-based active learning, to focus data acquisition on the most informative skill parameters. Additionally, ActivePusher seamlessly integrates with model-based kinodynamic planners, leveraging uncertainty estimates to bias control sampling toward more reliable actions. We evaluate our approach in both simulation and real-world environments, and demonstrate that it consistently improves data efficiency and achieves higher planning success rates in comparison to baseline methods. The source code is available at https://github.com/elpis-lab/ActivePusher.