# VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams

arXiv: https://arxiv.org/abs/2312.01407 · HTML: https://mlsanity.com/paper/2312.01407/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 63 of its 83 references that are in the map.
- First posted on arXiv 3 Dec 2023.

## Builds on (most cited first, 30 of 63)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video](https://mlsanity.com/paper/2012.12247/) (2020)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [Fast Dynamic Radiance Fields with Time-Aware Neural Voxels](https://mlsanity.com/paper/2205.15285/) (2022)
- [NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction](https://mlsanity.com/paper/2212.05231/) (2022)
- [MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures](https://mlsanity.com/paper/2208.00277/) (2022)
- [Neural Radiance Flow for 4D View Synthesis and Video Processing](https://mlsanity.com/paper/2012.09790/) (2020)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis](https://mlsanity.com/paper/2302.14859/) (2023)
- [Consistent Video Depth Estimation](https://mlsanity.com/paper/2004.15021/) (2020)
- [MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes](https://mlsanity.com/paper/2302.12249/) (2023)
- [Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering](https://mlsanity.com/paper/2211.11610/) (2022)
- [NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads](https://mlsanity.com/paper/2305.03027/) (2023)
- [TAVA: Template-free Animatable Volumetric Actors](https://mlsanity.com/paper/2206.08929/) (2022)
- [Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time](https://mlsanity.com/paper/2202.08614/) (2022)

## Built on by (most cited first, 22 of 22)

- [Robust Dual Gaussian Splatting for Immersive Human-centric Volumetric Videos](https://mlsanity.com/paper/2409.08353/) (2024)
- [V^3: Viewing Volumetric Videos on Mobiles via Streamable 2D Dynamic Gaussians](https://mlsanity.com/paper/2409.13648/) (2024)
- [4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video](https://mlsanity.com/paper/2503.18421/) (2025)
- [Compression of 3D Gaussian Splatting with Optimized Feature Planes and Standard Video Codecs](https://mlsanity.com/paper/2501.03399/) (2025)
- [GIFStream: 4D Gaussian-based Immersive Video with Feature Stream](https://mlsanity.com/paper/2505.07539/) (2025)
- [Implicit Gaussian Splatting with Efficient Multi-Level Tri-Plane Representation](https://mlsanity.com/paper/2408.10041/) (2024)
- [VRVVC: Variable-Rate NeRF-Based Volumetric Video Compression](https://mlsanity.com/paper/2412.11362/) (2024)
- [HPC: Hierarchical Progressive Coding Framework for Volumetric Video](https://mlsanity.com/paper/2407.09026/) (2024)
- [SqueezeMe: Mobile-Ready Distillation of Gaussian Full-Body Avatars](https://mlsanity.com/paper/2412.15171/) (2024)
- [Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streaming](https://mlsanity.com/paper/2503.08166/) (2025)
- [AdaGaR: Adaptive Gabor Representation for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2601.00796/) (2026)
- [Rate-aware Compression for NeRF-based Volumetric Video](https://mlsanity.com/paper/2411.05322/) (2024)
- [RePerformer: Immersive Human-centric Volumetric Videos from Playback to Photoreal Reperformance](https://mlsanity.com/paper/2503.12242/) (2025)
- [EvolvingGS: High-Fidelity Streamable Volumetric Video via Evolving 3D Gaussian Representation](https://mlsanity.com/paper/2503.05162/) (2025)
- [High-Fidelity Mobile Avatars with Pruned Local Blendshapes](https://mlsanity.com/paper/2605.01854/) (2026)
- [CWRNN-INVR: A Coupled WarpRNN based Implicit Neural Video Representation](https://mlsanity.com/paper/2604.06564/) (2026)
- [HGC-Avatar: Hierarchical Gaussian Compression for Streamable Dynamic 3D Avatars](https://mlsanity.com/paper/2510.16463/) (2025)
- [Constrained Dynamic Gaussian Splatting](https://mlsanity.com/paper/2602.03538/) (2026)
- [CATRF: Codec-Adaptive TriPlane Radiance Fields for Volumetric Content Delivery](https://mlsanity.com/paper/2605.18054/) (2026)
- [DLGStream: Dynamic Language-embedded Guassian Splatting for Open-vocabulary Enabled Free-viewpoint Video Streaming](https://mlsanity.com/paper/2606.28840/) (2026)
- [Tele360: Real-Time Feed-Forward Human Reconstruction from Sparse Unposed Cameras](https://mlsanity.com/paper/2609.15032/) (2026)
- [Racing in Volume with Flow Ensembles](https://mlsanity.com/paper/2609.16310/) (2026)

## Abstract (arXiv)

Neural Radiance Fields (NeRFs) excel in photorealistically rendering static scenes. However, rendering dynamic, long-duration radiance fields on ubiquitous devices remains challenging, due to data storage and computational constraints. In this paper, we introduce VideoRF, the first approach to enable real-time streaming and rendering of dynamic radiance fields on mobile platforms. At the core is a serialized 2D feature image stream representing the 4D radiance field all in one. We introduce a tailored training scheme directly applied to this 2D domain to impose the temporal and spatial redundancy of the feature image stream. By leveraging the redundancy, we show that the feature image stream can be efficiently compressed by 2D video codecs, which allows us to exploit video hardware accelerators to achieve real-time decoding. On the other hand, based on the feature image stream, we propose a novel rendering pipeline for VideoRF, which has specialized space mappings to query radiance properties efficiently. Paired with a deferred shading model, VideoRF has the capability of real-time rendering on mobile devices thanks to its efficiency. We have developed a real-time interactive player that enables online streaming and rendering of dynamic scenes, offering a seamless and immersive free-viewpoint experience across a range of devices, from desktops to mobile phones.