# Transformable Bottleneck Networks

arXiv: https://arxiv.org/abs/1904.06458 · HTML: https://mlsanity.com/paper/1904.06458/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2019.

## Built on by (most cited first, 23 of 23)

- [Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis](https://mlsanity.com/paper/2104.00677/) (2021)
- [Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior](https://mlsanity.com/paper/2303.14184/) (2023)
- [Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Rendering](https://mlsanity.com/paper/2109.07448/) (2021)
- [Unconstrained Scene Generation with Locally Conditioned Radiance Fields](https://mlsanity.com/paper/2104.00670/) (2021)
- [Control-NeRF: Editable Feature Volumes for Scene Rendering and Manipulation](https://mlsanity.com/paper/2204.10850/) (2022)
- [Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering](https://mlsanity.com/paper/2112.04312/) (2021)
- [PVA: Pixel-aligned Volumetric Avatars](https://mlsanity.com/paper/2101.02697/) (2021)
- [Fast and Explicit Neural View Synthesis](https://mlsanity.com/paper/2107.05775/) (2021)
- [Flow Guided Transformable Bottleneck Networks for Motion Retargeting](https://mlsanity.com/paper/2106.07771/) (2021)
- [Bridging Implicit and Explicit Geometric Transformation for Single-Image View Synthesis](https://mlsanity.com/paper/2209.07105/) (2022)
- [Learning Generalizable Light Field Networks from Few Images](https://mlsanity.com/paper/2207.11757/) (2022)
- [Unsupervised Novel View Synthesis from a Single Image](https://mlsanity.com/paper/2102.03285/) (2021)
- [Novel View Synthesis via Depth-guided Skip Connections](https://mlsanity.com/paper/2101.01619/) (2021)
- [GaussiGAN: Controllable Image Synthesis with 3D Gaussians from Unposed Silhouettes](https://mlsanity.com/paper/2106.13215/) (2021)
- [ViewNet: Unsupervised Viewpoint Estimation from Conditional Generation](https://mlsanity.com/paper/2212.00435/) (2022)
- [Novel View Video Prediction Using a Dual Representation](https://mlsanity.com/paper/2106.03956/) (2021)
- [STR-GQN: Scene Representation and Rendering for Unknown Cameras Based on Spatial Transformation Routing](https://mlsanity.com/paper/2108.03072/) (2021)
- [Appearance Editing with Free-viewpoint Neural Rendering](https://mlsanity.com/paper/2110.07674/) (2021)
- [Novel View Synthesis from a Single Image via Unsupervised learning](https://mlsanity.com/paper/2110.15569/) (2021)
- [Self-Supervised Equivariant Scene Synthesis from Video](https://mlsanity.com/paper/2102.00863/) (2021)
- [FaDIV-Syn: Fast Depth-Independent View Synthesis using Soft Masks and Implicit Blending](https://mlsanity.com/paper/2106.13139/) (2021)
- [Challenges and Solutions in DeepFakes](https://mlsanity.com/paper/2109.05397/) (2021)
- [Machine Learning for Detection of 3D Features using sparse X-ray data](https://mlsanity.com/paper/2206.02564/) (2022)

## Abstract (arXiv)

We propose a novel approach to performing fine-grained 3D manipulation of image content via a convolutional neural network, which we call the Transformable Bottleneck Network (TBN). It applies given spatial transformations directly to a volumetric bottleneck within our encoder-bottleneck-decoder architecture. Multi-view supervision encourages the network to learn to spatially disentangle the feature space within the bottleneck. The resulting spatial structure can be manipulated with arbitrary spatial transformations. We demonstrate the efficacy of TBNs for novel view synthesis, achieving state-of-the-art results on a challenging benchmark. We demonstrate that the bottlenecks produced by networks trained for this task contain meaningful spatial structure that allows us to intuitively perform a variety of image manipulations in 3D, well beyond the rigid transformations seen during training. These manipulations include non-uniform scaling, non-rigid warping, and combining content from different images. Finally, we extract explicit 3D structure from the bottleneck, performing impressive 3D reconstruction from a single input image.