Deep Convolutional Inverse Graphics Network
Not on the map. Why?
Abstract (arXiv)
This paper presents the Deep Convolution Inverse Graphics Network (DC-IGN), a model that learns an interpretable representation of images. This representation is disentangled with respect to transformations such as out-of-plane rotations and lighting variations. The DC-IGN model is composed of multiple layers of convolution and de-convolution operators and is trained using the Stochastic Gradient Variational Bayes (SGVB) algorithm. We propose a training procedure to encourage neurons in the graphics code layer to represent a specific transformation (e.g. pose or light). Given a single input image, our model can generate new images of the same object with variations in pose and lighting. We present qualitative and quantitative results of the model's efficacy at learning a 3D rendering engine.
Key facts. Cited by 121 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Mar 2015.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 37; 2022: 37; 2023: 21; 2024: 11; 2025: 13; 2026: 2.
121 papers.
Sort:
By area:
- 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
- 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
- 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
- 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
- 2021Neural Articulated Radiance Fieldcited by 196
- 2023StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Modelscited by 154
- 2021Illiterate DALL-E Learns to Composecited by 147
- 2021A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representationcited by 113
- 2022Disentangled Representation Learningcited by 111
- 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 99
- 2022GlanceNets: Interpretabile, Leak-proof Concept-based Modelscited by 73
- 2024Advances in 3D Generation: A Surveycited by 62
38 more
- 2021CDFI: Compression-Driven Network Design for Frame Interpolationcited by 55
- 2022On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligencecited by 50
- 2022Recent Advances for Quantum Neural Networks in Generative Learningcited by 39
- 2021DISSECT: Disentangled Simultaneous Explanations via Concept Traversalscited by 37
- 2023Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometrycited by 35
- 2021Video Autoencoder: self-supervised disentanglement of static 3D structure and motioncited by 34
- 2022Reversible Column Networkscited by 32
- 2023Leveraging sparse and shared feature activations for disentangled representation learningcited by 32
- 2022De-rendering 3D Objects in the Wildcited by 31
- 2021De-rendering the World's Revolutionary Artefactscited by 28
- 2021Where and What? Examining Interpretable Disentangled Representationscited by 27
- 2021Commutative Lie Group VAE for Disentanglement Learningcited by 27
- 2022Symmetry-Based Representations for Artificial and Biological General Intelligencecited by 26
- 2023GINA-3D: Learning to Generate Implicit Neural Assets in the Wildcited by 25
- 2021Deep Metric Learning for Few-Shot Image Classification: A Review of Recent Developmentscited by 23
- 2022DCI-ES: An Extended Disentanglement Framework with Connections to Identifiabilitycited by 23
- 2023Multifactor Sequential Disentanglement via Structured Koopman Autoencoderscited by 23
- 2022MutexMatch: Semi-Supervised Learning with Mutex-Based Consistency Regularizationcited by 22
- 2021Representing 3D Shapes with Probabilistic Directed Distance Fieldscited by 21
- 2021Learning disentangled representations via product manifold projectioncited by 20
- 2021Outdoor inverse rendering from a single image using multiview self-supervisioncited by 18
- 2022Homomorphism Autoencoder -- Learning Group Structured Representations from Observed Transitionscited by 18
- 2024TryOffDiff: Virtual-Try-Off via High-Fidelity Garment Reconstruction using Diffusion Modelscited by 18
- 2022SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Imagescited by 17
- 2023Grounded Object Centric Learningcited by 17
- 2021Cloud2Curve: Generation and Vectorization of Parametric Sketchescited by 16
- 2023Self-Supervised Disentanglement by Leveraging Structure in Data Augmentationscited by 16
- 2021Disentangling Representations of Text by Masking Transformerscited by 15
- 20213D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Facescited by 15
- 2023SAOR: Single-View Articulated Object Reconstructioncited by 15
- 2025Generative Models in Decision Making: A Surveycited by 14
- 2022NashAE: Disentangling Representations through Adversarial Covariance Minimizationcited by 13
- 2021The Role of Pretrained Representations for the OOD Generalization of Reinforcement Learning Agentscited by 12
- 2022An Empirical Study on Disentanglement of Negative-free Contrastive Learningcited by 12
- 2022Learning Conditional Variational Autoencoders with Missing Covariatescited by 10
- 2022Learning with Capsules: A Surveycited by 10
- 2023Sample and Predict Your Latent: Modality-free Sequential Disentanglement via Contrastive Estimationcited by 10
- 2023TUVF: Learning Generalizable Texture UV Radiance Fieldscited by 9
50 most cited of 121 All 121 (JSON)
- 2026IP Protection in the Era of Visual Generative AI: A Survey13 Aug 2026
- 2026Twincher: Bijective Representation Learning for Robust Inversion of Continuous Systems13 May 2026
- 2025What We Don't C: Manifold Disentanglement for Structured Discovery12 Nov 2025
- 2025IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks23 Oct 2025
- 2025Bounding Distributional Shifts in World Modeling through Novelty Detection8 Aug 2025
- 2025Disentangled representations of microscopy images25 Jun 2025
- 2025IndoorWorld: Integrating Physical Task Solving and Social Simulation in A Heterogeneous Multi-Agent Environment14 Jun 2025
- 2025Modular Machine Learning: An Indispensable Path towards New-Generation Large Language Models28 Apr 2025
- 2025A Controllable Appearance Representation for Flexible Transfer and Editing21 Apr 2025
- 2025Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research17 Apr 2025
- 2025Wavelet-based Variational Autoencoders for High-Resolution Image Generation16 Apr 2025
- 2025Sketch & Paint: Stroke-by-Stroke Evolution of Visual Artworks27 Feb 2025
38 more
- 2025Generative Models in Decision Making: A Survey24 Feb 2025
- 2025GPO-VAE: Modeling Explainable Gene Perturbation Responses utilizing GRN-Aligned Parameter Optimization31 Jan 2025
- 2025Transformer Vibration Forecasting for Advancing Rail Safety and Maintenance 4.020 Jan 2025
- 2024Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models20 Dec 2024
- 2024TryOffDiff: Virtual-Try-Off via High-Fidelity Garment Reconstruction using Diffusion Models27 Nov 2024
- 2024Disentangling Genotype and Environment Specific Latent Features for Improved Trait Prediction using a Compositional Autoencoder25 Oct 2024
- 2024Transferring disentangled representations: bridging the gap between synthetic and real images26 Sep 2024
- 2024Bayesian Inverse Graphics for Few-Shot Concept Learning12 Sep 2024
- 2024Rethinking Disentanglement under Dependent Factors of Variation13 Aug 2024
- 2024Hybrid Classical-Quantum architecture for vectorised image classification of hand-written sketches8 Jul 2024
- 2024Identifiable Object-Centric Representation Learning via Probabilistic Slot Attention11 Jun 2024
- 2024Towards Controllable Time Series Generation6 Mar 2024
- 2024Advances in 3D Generation: A Survey31 Jan 2024
- 2024GT-PCA: Effective and Interpretable Dimensionality Reduction with General Transform-Invariant Principal Component Analysis28 Jan 2024
- 2023Emergence and Function of Abstract Representations in Self-Supervised Transformers8 Dec 2023
- 2023DEVIAS: Learning Disentangled Video Representations of Action and Scene30 Nov 2023
- 2023Self-Supervised Disentanglement by Leveraging Structure in Data Augmentations15 Nov 2023
- 2023Generating by Understanding: Neural Visual Generation with Logical Symbol Groundings26 Oct 2023
- 2023Zero-Shot Transfer in Imitation Learning10 Oct 2023
- 2023Disentanglement Learning via Topology24 Aug 2023
- 2023StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models15 Aug 2023
- 2023Grounded Object Centric Learning18 Jul 2023
- 2023Capsule network with shortcut routing14 Jul 2023
- 2023Sample and Predict Your Latent: Modality-free Sequential Disentanglement via Contrastive Estimation25 May 2023
- 2023Inverse Rendering of Translucent Objects using Physical and Neural Renderers15 May 2023
- 2023TUVF: Learning Generalizable Texture UV Radiance Fields4 May 2023
- 2023Single-View View Synthesis with Self-Rectified Pseudo-Stereo19 Apr 2023
- 2023Leveraging sparse and shared feature activations for disentangled representation learning17 Apr 2023
- 2023Inferring Fluid Dynamics via Inverse Rendering10 Apr 2023
- 2023GINA-3D: Learning to Generate Implicit Neural Assets in the Wild4 Apr 2023
- 2023Multifactor Sequential Disentanglement via Structured Koopman Autoencoders30 Mar 2023
- 2023SAOR: Single-View Articulated Object Reconstruction23 Mar 2023
- 2023Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry15 Mar 2023
- 20233D Generative Model Latent Disentanglement via Local Eigenprojection24 Feb 2023
- 2023Structured Generative Models for Scene Understanding7 Feb 2023
- 2022MVTN: Learning Multi-View Transformations for 3D Understanding27 Dec 2022
- 2022Reversible Column Networks22 Dec 2022
- 2022SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Images18 Dec 2022
50 newest of 121 All 121 (JSON)
- 2023Zero-Shot Transfer in Imitation Learningcited by 0
- 2023Zero-Shot Transfer in Imitation Learning10 Oct 2023
- 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
- 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
- 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
- 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
- 2021Neural Articulated Radiance Fieldcited by 196
- 2021Illiterate DALL-E Learns to Composecited by 147
- 2021A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representationcited by 113
- 2022Disentangled Representation Learningcited by 111
- 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 99
- 2022GlanceNets: Interpretabile, Leak-proof Concept-based Modelscited by 73
- 2024Advances in 3D Generation: A Surveycited by 62
- 2021CDFI: Compression-Driven Network Design for Frame Interpolationcited by 55
12 most cited of 112 All 112 (JSON)
- 2026IP Protection in the Era of Visual Generative AI: A Survey13 Aug 2026
- 2026Twincher: Bijective Representation Learning for Robust Inversion of Continuous Systems13 May 2026
- 2025What We Don't C: Manifold Disentanglement for Structured Discovery12 Nov 2025
- 2025IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks23 Oct 2025
- 2025Disentangled representations of microscopy images25 Jun 2025
- 2025Modular Machine Learning: An Indispensable Path towards New-Generation Large Language Models28 Apr 2025
- 2025A Controllable Appearance Representation for Flexible Transfer and Editing21 Apr 2025
- 2025Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research17 Apr 2025
- 2025Wavelet-based Variational Autoencoders for High-Resolution Image Generation16 Apr 2025
- 2025Sketch & Paint: Stroke-by-Stroke Evolution of Visual Artworks27 Feb 2025
- 2025Generative Models in Decision Making: A Survey24 Feb 2025
- 2025GPO-VAE: Modeling Explainable Gene Perturbation Responses utilizing GRN-Aligned Parameter Optimization31 Jan 2025
12 newest of 112 All 112 (JSON)