# Adversarial Inverse Graphics Networks: Learning 2D-to-3D Lifting and Image-to-Image Translation from Unpaired Supervision

arXiv: https://arxiv.org/abs/1705.11166 · HTML: https://mlsanity.com/paper/1705.11166/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2017.

## Built on by (most cited first, 27 of 27)

- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [LOLNeRF: Learn from One Look](https://mlsanity.com/paper/2111.09996/) (2021)
- [HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recovery](https://mlsanity.com/paper/2304.05690/) (2023)
- [ElePose: Unsupervised 3D Human Pose Estimation by Predicting Camera Elevation and Learning Normalizing Flows on 2D Poses](https://mlsanity.com/paper/2112.07088/) (2021)
- [Unsupervised Human Pose Estimation through Transforming Shape Templates](https://mlsanity.com/paper/2105.04154/) (2021)
- [Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation](https://mlsanity.com/paper/2203.15293/) (2022)
- [PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision](https://mlsanity.com/paper/2203.15625/) (2022)
- [Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection](https://mlsanity.com/paper/2106.07852/) (2021)
- [Autoinverse: Uncertainty Aware Inversion of Neural Networks](https://mlsanity.com/paper/2208.13780/) (2022)
- [Deep Feedback Inverse Problem Solver](https://mlsanity.com/paper/2101.07719/) (2021)
- [Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry](https://mlsanity.com/paper/2108.07777/) (2021)
- [On Triangulation as a Form of Self-Supervision for 3D Human Pose Estimation](https://mlsanity.com/paper/2203.15865/) (2022)
- [Training Generative Adversarial Networks in One Stage](https://mlsanity.com/paper/2103.00430/) (2021)
- [3D Human Pose Estimation Using M"obius Graph Convolutional Networks](https://mlsanity.com/paper/2203.10554/) (2022)
- [Non-Local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimation](https://mlsanity.com/paper/2204.01971/) (2022)
- [LInKs "Lifting Independent Keypoints" -- Partial Pose Lifting for Occlusion Handling with Improved Accuracy in 2D-3D Human Pose Estimation](https://mlsanity.com/paper/2309.07243/) (2023)
- [Learning Unsupervised Cross-domain Image-to-Image Translation Using a Shared Discriminator](https://mlsanity.com/paper/2102.04699/) (2021)
- [Learning Temporal 3D Human Pose Estimation with Pseudo-Labels](https://mlsanity.com/paper/2110.07578/) (2021)
- [One Ring to Rule Them All: a simple solution to multi-view 3D-Reconstruction of shapes with unknown BRDF via a small Recurrent ResNet](https://mlsanity.com/paper/2104.05014/) (2021)
- [Heuristic Weakly Supervised 3D Human Pose Estimation](https://mlsanity.com/paper/2105.10996/) (2021)
- [SVMAC: Unsupervised 3D Human Pose Estimation from a Single Image with Single-view-multi-angle Consistency](https://mlsanity.com/paper/2106.05616/) (2021)
- [Quater-GCN: Enhancing 3D Human Pose Estimation with Orientation and Semi-supervised Training](https://mlsanity.com/paper/2404.19279/) (2024)
- [3D Human Pose Estimation Based on 2D-3D Consistency with Synchronized Adversarial Training](https://mlsanity.com/paper/2106.04274/) (2021)
- [EPOCH: Jointly Estimating the 3D Pose of Cameras and Humans](https://mlsanity.com/paper/2406.19726/) (2024)
- [A Survey on Adversarial Image Synthesis](https://mlsanity.com/paper/2106.16056/) (2021)
- [Unsupervised Style-based Explicit 3D Face Reconstruction from Single Image](https://mlsanity.com/paper/2304.12455/) (2023)
- [Learning Human Motion from Monocular Videos via Cross-Modal Manifold Alignment](https://mlsanity.com/paper/2404.09499/) (2024)

## Abstract (arXiv)

Researchers have developed excellent feed-forward models that learn to map images to desired outputs, such as to the images' latent factors, or to other images, using supervised learning. Learning such mappings from unlabelled data, or improving upon supervised models by exploiting unlabelled data, remains elusive. We argue that there are two important parts to learning without annotations: (i) matching the predictions to the input observations, and (ii) matching the predictions to known priors. We propose Adversarial Inverse Graphics networks (AIGNs): weakly supervised neural network models that combine feedback from rendering their predictions, with distribution matching between their predictions and a collection of ground-truth factors. We apply AIGNs to 3D human pose estimation and 3D structure and egomotion estimation, and outperform models supervised by only paired annotations. We further apply AIGNs to facial image transformation using super-resolution and inpainting renderers, while deliberately adding biases in the ground-truth datasets. Our model seamlessly incorporates such biases, rendering input faces towards young, old, feminine, masculine or Tom Cruise-like equivalents (depending on the chosen bias), or adding lip and nose augmentations while inpainting concealed lips and noses.