# X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation

arXiv: https://arxiv.org/abs/2010.00450 · HTML: https://mlsanity.com/paper/2010.00450/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2020.

## Built on by (most cited first, 30 of 31)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis](https://mlsanity.com/paper/2312.16812/) (2023)
- [DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks](https://mlsanity.com/paper/2103.03231/) (2021)
- [DynIBaR: Neural Dynamic Image-Based Rendering](https://mlsanity.com/paper/2211.11082/) (2022)
- [MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffolds](https://mlsanity.com/paper/2405.17421/) (2024)
- [HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling](https://mlsanity.com/paper/2301.02238/) (2023)
- [Editable Free-viewpoint Video Using a Layered Neural Representation](https://mlsanity.com/paper/2104.14786/) (2021)
- [Neural Point Catacaustics for Novel-View Synthesis of Reflections](https://mlsanity.com/paper/2301.01087/) (2023)
- [Neural Residual Radiance Fields for Streamably Free-Viewpoint Videos](https://mlsanity.com/paper/2304.04452/) (2023)
- [R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis](https://mlsanity.com/paper/2203.17261/) (2022)
- [Local-to-Global Registration for Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2211.11505/) (2022)
- [Free-viewpoint Indoor Neural Relighting from Multi-view Stereo](https://mlsanity.com/paper/2106.13299/) (2021)
- [Relighting Neural Radiance Fields with Shadow and Highlight Hints](https://mlsanity.com/paper/2308.13404/) (2023)
- [FLAME-in-NeRF : Neural control of Radiance Fields for Free View Face Animation](https://mlsanity.com/paper/2108.04913/) (2021)
- [Eikonal Fields for Refractive Novel-View Synthesis](https://mlsanity.com/paper/2202.00948/) (2022)
- [3D Cinemagraphy from a Single Image](https://mlsanity.com/paper/2303.05724/) (2023)
- [Single Image Super-Resolution via a Dual Interactive Implicit Neural Network](https://mlsanity.com/paper/2210.12593/) (2022)
- [Active Exploration for Neural Global Illumination of Variable Scenes](https://mlsanity.com/paper/2203.08272/) (2022)
- [Spatial Attention-based Implicit Neural Representation for Arbitrary Reduction of MRI Slice Spacing](https://mlsanity.com/paper/2205.11346/) (2022)
- [VIINTER: View Interpolation with Implicit Neural Representations of Images](https://mlsanity.com/paper/2211.00722/) (2022)
- [Learning Continuous Implicit Representation for Near-Periodic Patterns](https://mlsanity.com/paper/2208.12278/) (2022)
- [Frame Interpolation for Dynamic Scenes with Implicit Flow Encoding](https://mlsanity.com/paper/2209.13284/) (2022)
- [Rethinking Implicit Neural Representations for Vision Learners](https://mlsanity.com/paper/2211.12040/) (2022)
- [Light Field Implicit Representation for Flexible Resolution Reconstruction](https://mlsanity.com/paper/2112.00185/) (2021)
- [Detachable Novel Views Synthesis of Dynamic Scenes Using Distribution-Driven Neural Radiance Fields](https://mlsanity.com/paper/2301.00411/) (2023)
- [MiNL: Micro-images based Neural Representation for Light Fields](https://mlsanity.com/paper/2209.08277/) (2022)
- [Implicit View-Time Interpolation of Stereo Videos using Multi-Plane Disparities and Non-Uniform Coordinates](https://mlsanity.com/paper/2303.17181/) (2023)
- [Time-Archival Camera Virtualization for Sports and Visual Performances](https://mlsanity.com/paper/2602.15181/) (2026)
- [Stereo-Knowledge Distillation from dpMV to Dual Pixels for Light Field Video Reconstruction](https://mlsanity.com/paper/2405.11823/) (2024)

## Abstract (arXiv)

We suggest to represent an X-Field -a set of 2D images taken across different view, time or illumination conditions, i.e., video, light field, reflectance fields or combinations thereof-by learning a neural network (NN) to map their view, time or light coordinates to 2D images. Executing this NN at new coordinates results in joint view, time or light interpolation. The key idea to make this workable is a NN that already knows the "basic tricks" of graphics (lighting, 3D projection, occlusion) in a hard-coded and differentiable form. The NN represents the input to that rendering as an implicit map, that for any view, time, or light coordinate and for any pixel can quantify how it will move if view, time or light coordinates change (Jacobian of pixel position with respect to view, time, illumination, etc.). Our X-Field representation is trained for one scene within minutes, leading to a compact set of trainable parameters and hence real-time navigation in view, time and illumination.