# Learning to Simulate Tree-Branch Dynamics for Manipulation

arXiv: https://arxiv.org/abs/2306.03410 · HTML: https://mlsanity.com/paper/2306.03410/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 6 Jun 2023.

## Builds on (most cited first, 11 of 11)

- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience](https://mlsanity.com/paper/1810.05687/) (2018)
- [DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data](https://mlsanity.com/paper/1912.04302/) (2019)
- [Inferring the Material Properties of Granular Media for Robotic Tasks](https://mlsanity.com/paper/2003.08032/) (2020)
- [Probabilistic Inference of Simulation Parameters via Parallel Differentiable Simulation](https://mlsanity.com/paper/2109.08815/) (2021)
- [A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulation](https://mlsanity.com/paper/2112.05068/) (2021)
- [Three Dimensional Reconstruction of Botanical Trees with Simulatable Geometry](https://mlsanity.com/paper/1812.08849/) (2018)
- [Tree Reconstruction using Topology Optimisation](https://mlsanity.com/paper/2205.13192/) (2022)

## Built on by (most cited first, 4 of 4)

- [Robust Flower Cluster Matching Using The Unscented Transform](https://mlsanity.com/paper/2503.20631/) (2025)
- [Deformable Cluster Manipulation via Whole-Arm Policy Learning](https://mlsanity.com/paper/2507.17085/) (2025)
- [Planning-Query-Guided Model Generation for Model-Based Deformable Object Manipulation](https://mlsanity.com/paper/2508.19199/) (2025)
- [SILO: Simulation-in-the-Loop Sim-to-Real Transfer for Multi-Stage Cable Routing](https://mlsanity.com/paper/2607.04616/) (2026)

## Abstract (arXiv)

We propose to use a simulation driven inverse inference approach to model the dynamics of tree branches under manipulation. Learning branch dynamics and gaining the ability to manipulate deformable vegetation can help with occlusion-prone tasks, such as fruit picking in dense foliage, as well as moving overhanging vines and branches for navigation in dense vegetation. The underlying deformable tree geometry is encapsulated as coarse spring abstractions executed on parallel, non-differentiable simulators. The implicit statistical model defined by the simulator, reference trajectories obtained by actively probing the ground truth, and the Bayesian formalism, together guide the spring parameter posterior density estimation. Our non-parametric inference algorithm, based on Stein Variational Gradient Descent, incorporates biologically motivated assumptions into the inference process as neural network driven learnt joint priors; moreover, it leverages the finite difference scheme for gradient approximations. Real and simulated experiments confirm that our model can predict deformation trajectories, quantify the estimation uncertainty, and it can perform better when base-lined against other inference algorithms, particularly from the Monte Carlo family. The model displays strong robustness properties in the presence of heteroscedastic sensor noise; furthermore, it can generalise to unseen grasp locations.