mlsanityData to 7 Oct 2026

MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images

arXiv 2008.06534 ↗First posted 14 Aug 2020cs.CVcited by 64

Not on the map. Why?

Abstract (arXiv)

We introduce a method to convert stereo 360deg (omnidirectional stereo) imagery into a layered, multi-sphere image representation for six degree-of-freedom (6DoF) rendering. Stereo 360deg imagery can be captured from multi-camera systems for virtual reality (VR), but lacks motion parallax and correct-in-all-directions disparity cues. Together, these can quickly lead to VR sickness when viewing content. One solution is to try and generate a format suitable for 6DoF rendering, such as by estimating depth. However, this raises questions as to how to handle disoccluded regions in dynamic scenes. Our approach is to simultaneously learn depth and disocclusions via a multi-sphere image representation, which can be rendered with correct 6DoF disparity and motion parallax in VR. This significantly improves comfort for the viewer, and can be inferred and rendered in real time on modern GPU hardware. Together, these move towards making VR video a more comfortable immersive medium.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 15; 2023: 13; 2024: 11; 2025: 3; 2026: 4.

64 papers.

Sort:

  1. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  2. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  3. 2021Advances in Neural Renderingcited by 365
  4. 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 341
  5. 2022MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architecturescited by 308
  6. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  7. 2023MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenescited by 215
  8. 2021Point-Based Neural Rendering with Per-View Optimizationcited by 188
  9. 2023HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Samplingcited by 173
  10. 2021BACON: Band-limited Coordinate Networks for Multiscale Scene Representationcited by 142
  11. 2022Streaming Radiance Fields for 3D Video Synthesiscited by 118
  12. 2021Neural Lumigraph Renderingcited by 105
38 more
  1. 2021T"oRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesiscited by 88
  2. 2022ReLU Fields: The Little Non-linearity That Couldcited by 87
  3. 2021FoV-NeRF: Foveated Neural Radiance Fields for Virtual Realitycited by 79
  4. 2021ACORN: Adaptive Coordinate Networks for Neural Scene Representationcited by 64
  5. 2021360MonoDepth: High-Resolution 360deg Monocular Depth Estimationcited by 60
  6. 2021Full Surround Monodepth from Multiple Camerascited by 41
  7. 2021Pano3D: A Holistic Benchmark and a Solid Baseline for 360o Depth Estimationcited by 37
  8. 2023Real-Time Neural Rasterization for Large Scenescited by 37
  9. 2021Fast Training of Neural Lumigraph Representations using Meta Learningcited by 29
  10. 2021HVH: Learning a Hybrid Neural Volumetric Representation for Dynamic Hair Performance Capturecited by 29
  11. 2024Splatter-360: Generalizable 360circ Gaussian Splatting for Wide-baseline Panoramic Imagescited by 23
  12. 2024PanSplat: 4K Panorama Synthesis with Feed-Forward Gaussian Splattingcited by 23
  13. 2023PanoGRF: Generalizable Spherical Radiance Fields for Wide-baseline Panoramascited by 22
  14. 2022360Roam: Real-Time Indoor Roaming Using Geometry-Aware 360^circ Radiance Fieldscited by 20
  15. 2023Local Implicit Ray Function for Generalizable Radiance Field Representationcited by 19
  16. 2024ODGS: 3D Scene Reconstruction from Omnidirectional Images with 3D Gaussian Splattingscited by 19
  17. 2021Layout-Guided Novel View Synthesis from a Single Indoor Panoramacited by 16
  18. 2023ECNR: Efficient Compressive Neural Representation of Time-Varying Volumetric Datasetscited by 16
  19. 2024Quark: Real-time, High-resolution, and General Neural View Synthesiscited by 16
  20. 2023Tiled Multiplane Images for Practical 3D Photographycited by 14
  21. 2023Point-DynRF: Point-based Dynamic Radiance Fields from a Monocular Videocited by 12
  22. 2025Neural Radiance Fields for the Real World: A Surveycited by 12
  23. 2022Stereo Magnification with Multi-Layer Imagescited by 11
  24. 2022Casual Indoor HDR Radiance Capture from Omnidirectional Imagescited by 11
  25. 2022Enhancement of Novel View Synthesis Using Omnidirectional Image Completioncited by 9
  26. 2023Volumetric Rendering with Baked Quadrature Fieldscited by 9
  27. 2022Casual 6-DoF: free-viewpoint panorama using a handheld 360 cameracited by 7
  28. 2023INV: Towards Streaming Incremental Neural Videoscited by 7
  29. 2024OmniSDF: Scene Reconstruction using Omnidirectional Signed Distance Functions and Adaptive Binoctreescited by 6
  30. 2024IE-NeRF: Inpainting Enhanced Neural Radiance Fields in the Wildcited by 6
  31. 2024Neural Light Spheres for Implicit Image Stitching and View Synthesiscited by 6
  32. 2022OmniSyn: Synthesizing 360 Videos with Wide-baseline Panoramascited by 5
  33. 2024OmniLocalRF: Omnidirectional Local Radiance Fields from Dynamic Videoscited by 5
  34. 2026VGGT-360: Geometry-Consistent Zero-Shot Panoramic Depth Estimationcited by 5
  35. 2022Learning Dynamic View Synthesis With Few RGBD Camerascited by 3
  36. 2022Hybrid Skip: A Biologically Inspired Skip Connection for the UNet Architecturecited by 3
  37. 2022Progressive Multi-scale Light Field Networkscited by 3
  38. 2025ProSplat: Improved Feed-Forward 3D Gaussian Splatting for Wide-Baseline Sparse Viewscited by 3

50 most cited of 64 All 64 (JSON)