# Bayesian Inverse Graphics for Few-Shot Concept Learning

arXiv: https://arxiv.org/abs/2409.08351 · HTML: https://mlsanity.com/paper/2409.08351/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 55 references that are in the map.
- First posted on arXiv 12 Sep 2024.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [The Hardware Lottery](https://mlsanity.com/paper/2009.06489/) (2020)
- [OnePose: One-Shot Object Pose Estimation without CAD Models](https://mlsanity.com/paper/2205.12257/) (2022)
- [Equinox: neural networks in JAX via callable PyTrees and filtered transformations](https://mlsanity.com/paper/2111.00254/) (2021)
- [Deep Convolutional Inverse Graphics Network](https://mlsanity.com/paper/1503.03167/) (2015)
- [TensorFlow Distributions](https://mlsanity.com/paper/1711.10604/) (2017)
- [Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols](https://mlsanity.com/paper/1502.03143/) (2015)
- [LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation](https://mlsanity.com/paper/1912.00416/) (2019)
- [FS6D: Few-Shot 6D Pose Estimation of Novel Objects](https://mlsanity.com/paper/2203.14628/) (2022)
- [The Omniglot challenge: a 3-year progress report](https://mlsanity.com/paper/1902.03477/) (2019)
- [Bayesian Workflow](https://mlsanity.com/paper/2011.01808/) (2020)
- [3DP3: 3D Scene Perception via Probabilistic Programming](https://mlsanity.com/paper/2111.00312/) (2021)
- [I Find Your Lack of Uncertainty in Computer Vision Disturbing](https://mlsanity.com/paper/2104.08188/) (2021)
- [Differentiable Visual Computing for Inverse Problems and Machine Learning](https://mlsanity.com/paper/2312.04574/) (2023)

## Built on by (most cited first, 2 of 2)

- [Bayesian Inverse Physics for Neuro-Symbolic Robot Learning](https://mlsanity.com/paper/2506.08756/) (2025)
- [Differentiable Inverse Graphics for Zero-shot Scene Reconstruction and Robot Grasping](https://mlsanity.com/paper/2602.05029/) (2026)

## Abstract (arXiv)

Humans excel at building generalizations of new concepts from just one single example. Contrary to this, current computer vision models typically require large amount of training samples to achieve a comparable accuracy. In this work we present a Bayesian model of perception that learns using only minimal data, a prototypical probabilistic program of an object. Specifically, we propose a generative inverse graphics model of primitive shapes, to infer posterior distributions over physically consistent parameters from one or several images. We show how this representation can be used for downstream tasks such as few-shot classification and pose estimation. Our model outperforms existing few-shot neural-only classification algorithms and demonstrates generalization across varying lighting conditions, backgrounds, and out-of-distribution shapes. By design, our model is uncertainty-aware and uses our new differentiable renderer for optimizing global scene parameters through gradient descent, sampling posterior distributions over object parameters with Markov Chain Monte Carlo (MCMC), and using a neural based likelihood function.