# Fine-Tuning Generative Models as an Inference Method for Robotic Tasks

arXiv: https://arxiv.org/abs/2310.12862 · HTML: https://mlsanity.com/paper/2310.12862/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 19 Oct 2023.

## Builds on (most cited first, 24 of 24)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [3D Shape Generation and Completion through Point-Voxel Diffusion](https://mlsanity.com/paper/2104.03670/) (2021)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning](https://mlsanity.com/paper/1708.02596/) (2017)
- [VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning](https://mlsanity.com/paper/1910.08348/) (2019)
- [Preparing for the Unknown: Learning a Universal Policy with Online System Identification](https://mlsanity.com/paper/1702.02453/) (2017)
- [PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention](https://mlsanity.com/paper/1810.05591/) (2018)
- [Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Study](https://mlsanity.com/paper/1404.3785/) (2014)
- [BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators](https://mlsanity.com/paper/1906.01728/) (2019)
- [Real-Time Grasp Detection Using Convolutional Neural Networks](https://mlsanity.com/paper/1412.3128/) (2014)
- [Soft-IntroVAE: Analyzing and Improving the Introspective Variational Autoencoder](https://mlsanity.com/paper/2012.13253/) (2020)
- [Learning a Structured Latent Space for Unsupervised Point Cloud Completion](https://mlsanity.com/paper/2203.15580/) (2022)
- [Robust Grasp Planning Over Uncertain Shape Completions](https://mlsanity.com/paper/1903.00645/) (2019)
- [Reaching Through Latent Space: From Joint Statistics to Path Planning in Manipulation](https://mlsanity.com/paper/2210.11779/) (2022)
- [Ab Initio Particle-based Object Manipulation](https://mlsanity.com/paper/2107.08865/) (2021)
- [Simulation-based Bayesian inference for multi-fingered robotic grasping](https://mlsanity.com/paper/2109.14275/) (2021)
- [Bayesian and Neural Inference on LSTM-based Object Recognition from Tactile and Kinesthetic Information](https://mlsanity.com/paper/2306.06423/) (2023)

## Built on by (most cited first, 4 of 4)

- [Can Large Language Models Unlock Novel Scientific Research Ideas?](https://mlsanity.com/paper/2409.06185/) (2024)
- [Generative AI Agents in Autonomous Machines: A Safety Perspective](https://mlsanity.com/paper/2410.15489/) (2024)
- [Enhancing Generative Molecular Design via Uncertainty-guided Fine-tuning of Variational Autoencoders](https://mlsanity.com/paper/2405.20573/) (2024)
- [Incremental Few-Shot Adaptation for Non-Prehensile Object Manipulation using Parallelizable Physics Simulators](https://mlsanity.com/paper/2409.13228/) (2024)

## Abstract (arXiv)

Adaptable models could greatly benefit robotic agents operating in the real world, allowing them to deal with novel and varying conditions. While approaches such as Bayesian inference are well-studied frameworks for adapting models to evidence, we build on recent advances in deep generative models which have greatly affected many areas of robotics. Harnessing modern GPU acceleration, we investigate how to quickly adapt the sample generation of neural network models to observations in robotic tasks. We propose a simple and general method that is applicable to various deep generative models and robotic environments. The key idea is to quickly fine-tune the model by fitting it to generated samples matching the observed evidence, using the cross-entropy method. We show that our method can be applied to both autoregressive models and variational autoencoders, and demonstrate its usability in object shape inference from grasping, inverse kinematics calculation, and point cloud completion.