# Incremental Few-Shot Adaptation for Non-Prehensile Object Manipulation using Parallelizable Physics Simulators

arXiv: https://arxiv.org/abs/2409.13228 · HTML: https://mlsanity.com/paper/2409.13228/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 20 of 20)

- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Deep Visual Foresight for Planning Robot Motion](https://mlsanity.com/paper/1610.00696/) (2016)
- [Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience](https://mlsanity.com/paper/1810.05687/) (2018)
- [Sample-efficient Cross-Entropy Method for Real-time Planning](https://mlsanity.com/paper/2008.06389/) (2020)
- [Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL](https://mlsanity.com/paper/1812.07671/) (2018)
- [Scalable Differentiable Physics for Learning and Control](https://mlsanity.com/paper/2007.02168/) (2020)
- [STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation](https://mlsanity.com/paper/2104.13542/) (2021)
- [Predictive Sampling: Real-time Behaviour Synthesis with MuJoCo](https://mlsanity.com/paper/2212.00541/) (2022)
- [Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulation](https://mlsanity.com/paper/2206.11403/) (2022)
- [ASID: Active Exploration for System Identification in Robotic Manipulation](https://mlsanity.com/paper/2404.12308/) (2024)
- [DiffSDFSim: Differentiable Rigid-Body Dynamics With Implicit Shapes](https://mlsanity.com/paper/2111.15318/) (2021)
- [Non-prehensile Planar Manipulation via Trajectory Optimization with Complementarity Constraints](https://mlsanity.com/paper/2109.13145/) (2021)
- [Dual Online Stein Variational Inference for Control and Dynamics](https://mlsanity.com/paper/2103.12890/) (2021)
- [Model-Based Generalization Under Parameter Uncertainty Using Path Integral Control](https://mlsanity.com/paper/2006.03106/) (2020)
- [Uncertainty Averse Pushing with Model Predictive Path Integral Control](https://mlsanity.com/paper/1710.04005/) (2017)
- [Accurate Vision-based Manipulation through Contact Reasoning](https://mlsanity.com/paper/1911.03112/) (2019)
- [Fine-Tuning Generative Models as an Inference Method for Robotic Tasks](https://mlsanity.com/paper/2310.12862/) (2023)
- [Self-Adapting Recurrent Models for Object Pushing from Learning in Simulation](https://mlsanity.com/paper/2007.13421/) (2020)
- [Physics-Based Rigid Body Object Tracking and Friction Filtering From RGB-D Videos](https://mlsanity.com/paper/2309.15703/) (2023)

## Built on by (most cited first, 1 of 1)

- [PIN-WM: Learning Physics-INformed World Models for Non-Prehensile Manipulation](https://mlsanity.com/paper/2504.16693/) (2025)

## Abstract (arXiv)

Few-shot adaptation is an important capability for intelligent robots that perform tasks in open-world settings such as everyday environments or flexible production. In this paper, we propose a novel approach for non-prehensile manipulation which incrementally adapts a physics-based dynamics model for model-predictive control (MPC). The model prediction is aligned with a few examples of robot-object interactions collected with the MPC. This is achieved by using a parallelizable rigid-body physics simulation as dynamic world model and sampling-based optimization of the model parameters. In turn, the optimized dynamics model can be used for MPC using efficient sampling-based optimization. We evaluate our few-shot adaptation approach in object pushing experiments in simulation and with a real robot.