# Moving in a 360 World: Synthesizing Panoramic Parallaxes from a Single Panorama

arXiv: https://arxiv.org/abs/2106.10859 · HTML: https://mlsanity.com/paper/2106.10859/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Learning Category-Specific Mesh Reconstruction from Image Collections](https://mlsanity.com/paper/1803.07549/) (2018)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [3D Photography using Context-aware Layered Depth Inpainting](https://mlsanity.com/paper/2004.04727/) (2020)
- [Pushing the Boundaries of View Extrapolation with Multiplane Images](https://mlsanity.com/paper/1905.00413/) (2019)
- [Neural Rerendering in the Wild](https://mlsanity.com/paper/1904.04290/) (2019)
- [Learning the Depths of Moving People by Watching Frozen People](https://mlsanity.com/paper/1904.11111/) (2019)
- [HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation](https://mlsanity.com/paper/1901.03861/) (2019)
- [Revealing Scenes by Inverting Structure from Motion Reconstructions](https://mlsanity.com/paper/1904.03303/) (2019)
- [Crowdsampling the Plenoptic Function](https://mlsanity.com/paper/2007.15194/) (2020)
- [Multi-view 3D Models from Single Images with a Convolutional Network](https://mlsanity.com/paper/1511.06702/) (2015)
- [Meshlet Priors for 3D Mesh Reconstruction](https://mlsanity.com/paper/2001.01744/) (2020)

## Built on by (most cited first, 19 of 19)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models](https://mlsanity.com/paper/2303.11989/) (2023)
- [Removing Objects From Neural Radiance Fields](https://mlsanity.com/paper/2212.11966/) (2022)
- [PERF: Panoramic Neural Radiance Field from a Single Panorama](https://mlsanity.com/paper/2310.16831/) (2023)
- [Deep Learning for Omnidirectional Vision: A Survey and New Perspectives](https://mlsanity.com/paper/2205.10468/) (2022)
- [Balanced Spherical Grid for Egocentric View Synthesis](https://mlsanity.com/paper/2303.12408/) (2023)
- [A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision](https://mlsanity.com/paper/2502.10444/) (2025)
- [360FusionNeRF: Panoramic Neural Radiance Fields with Joint Guidance](https://mlsanity.com/paper/2209.14265/) (2022)
- [ODGS: 3D Scene Reconstruction from Omnidirectional Images with 3D Gaussian Splattings](https://mlsanity.com/paper/2410.20686/) (2024)
- [Pano2Room: Novel View Synthesis from a Single Indoor Panorama](https://mlsanity.com/paper/2408.11413/) (2024)
- [Casual Indoor HDR Radiance Capture from Omnidirectional Images](https://mlsanity.com/paper/2208.07903/) (2022)
- [Enhancement of Novel View Synthesis Using Omnidirectional Image Completion](https://mlsanity.com/paper/2203.09957/) (2022)
- [PointRegGPT: Boosting 3D Point Cloud Registration using Generative Point-Cloud Pairs for Training](https://mlsanity.com/paper/2407.14054/) (2024)
- [ErpGS: Equirectangular Image Rendering enhanced with 3D Gaussian Regularization](https://mlsanity.com/paper/2505.19883/) (2025)
- [OmniVoxel: A Fast and Precise Reconstruction Method of Omnidirectional Neural Radiance Field](https://mlsanity.com/paper/2208.06335/) (2022)
- [MaGRITTe: Manipulative and Generative 3D Realization from Image, Topview and Text](https://mlsanity.com/paper/2404.00345/) (2024)
- [You Need a Transition Plane: Bridging Continuous Panoramic 3D Reconstruction with Perspective Gaussian Splatting](https://mlsanity.com/paper/2504.09062/) (2025)
- [Enhancing Perception and Immersion in Pre-Captured Environments through Learning-Based Eye Height Adaptation](https://mlsanity.com/paper/2308.13042/) (2023)
- [MSI-NeRF: Linking Omni-Depth with View Synthesis through Multi-Sphere Image aided Generalizable Neural Radiance Field](https://mlsanity.com/paper/2403.10840/) (2024)

## Abstract (arXiv)

We present Omnidirectional Neural Radiance Fields (OmniNeRF), the first method to the application of parallax-enabled novel panoramic view synthesis. Recent works for novel view synthesis focus on perspective images with limited field-of-view and require sufficient pictures captured in a specific condition. Conversely, OmniNeRF can generate panorama images for unknown viewpoints given a single equirectangular image as training data. To this end, we propose to augment the single RGB-D panorama by projecting back and forth between a 3D world and different 2D panoramic coordinates at different virtual camera positions. By doing so, we are able to optimize an Omnidirectional Neural Radiance Field with visible pixels collecting from omnidirectional viewing angles at a fixed center for the estimation of new viewing angles from varying camera positions. As a result, the proposed OmniNeRF achieves convincing renderings of novel panoramic views that exhibit the parallax effect. We showcase the effectiveness of each of our proposals on both synthetic and real-world datasets.