# JointRF: End-to-End Joint Optimization for Dynamic Neural Radiance Field Representation and Compression

arXiv: https://arxiv.org/abs/2405.14452 · HTML: https://mlsanity.com/paper/2405.14452/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 17 references that are in the map.
- First posted on arXiv 23 May 2024.

## Builds on (most cited first, 17 of 17)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [Fast Dynamic Radiance Fields with Time-Aware Neural Voxels](https://mlsanity.com/paper/2205.15285/) (2022)
- [Neural Radiance Flow for 4D View Synthesis and Video Processing](https://mlsanity.com/paper/2012.09790/) (2020)
- [MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes](https://mlsanity.com/paper/2302.12249/) (2023)
- [Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering](https://mlsanity.com/paper/2211.11610/) (2022)
- [HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion](https://mlsanity.com/paper/2305.06356/) (2023)
- [Streaming Radiance Fields for 3D Video Synthesis](https://mlsanity.com/paper/2210.14831/) (2022)
- [DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering](https://mlsanity.com/paper/2307.10173/) (2023)
- [Neural Residual Radiance Fields for Streamably Free-Viewpoint Videos](https://mlsanity.com/paper/2304.04452/) (2023)
- [Compressing Volumetric Radiance Fields to 1 MB](https://mlsanity.com/paper/2211.16386/) (2022)
- [Factor Fields: A Unified Framework for Neural Fields and Beyond](https://mlsanity.com/paper/2302.01226/) (2023)
- [Efficient Dynamic-NeRF Based Volumetric Video Coding with Rate Distortion Optimization](https://mlsanity.com/paper/2402.01380/) (2024)
- [ECRF: Entropy-Constrained Neural Radiance Fields Compression with Frequency Domain Optimization](https://mlsanity.com/paper/2311.14208/) (2023)
- [FPO++: Efficient Encoding and Rendering of Dynamic Neural Radiance Fields by Analyzing and Enhancing Fourier PlenOctrees](https://mlsanity.com/paper/2310.20710/) (2023)

## Built on by (most cited first, 13 of 13)

- [4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video](https://mlsanity.com/paper/2503.18421/) (2025)
- [HPC: Hierarchical Progressive Coding Framework for Volumetric Video](https://mlsanity.com/paper/2407.09026/) (2024)
- [Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streaming](https://mlsanity.com/paper/2503.08166/) (2025)
- [Rate-aware Compression for NeRF-based Volumetric Video](https://mlsanity.com/paper/2411.05322/) (2024)
- [D-FCGS: Feedforward Compression of Dynamic Gaussian Splatting for Free-Viewpoint Videos](https://mlsanity.com/paper/2507.05859/) (2025)
- [4D-MoDe: Towards Editable and Scalable Volumetric Streaming via Motion-Decoupled 4D Gaussian Compression](https://mlsanity.com/paper/2509.17506/) (2025)
- [AlignGS: Aligning Geometry and Semantics for Robust Indoor Reconstruction from Sparse Views](https://mlsanity.com/paper/2510.07839/) (2025)
- [PrismGS: Physically-Grounded Anti-Aliasing for High-Fidelity Large-Scale 3D Gaussian Splatting](https://mlsanity.com/paper/2510.07830/) (2025)
- [GenSplatCodec: Feed-Forward Gaussian Splatting Compression via One-Step Diffusion](https://mlsanity.com/paper/2607.24403/) (2026)
- [MultiEgo: A Multi-View Egocentric Video Dataset for 4D Scene Reconstruction](https://mlsanity.com/paper/2512.11301/) (2025)
- [Constrained Dynamic Gaussian Splatting](https://mlsanity.com/paper/2602.03538/) (2026)
- [GeoRect4D: Geometry-Compatible Generative Rectification for Dynamic Sparse-View 3D Reconstruction](https://mlsanity.com/paper/2604.20784/) (2026)
- [CATRF: Codec-Adaptive TriPlane Radiance Fields for Volumetric Content Delivery](https://mlsanity.com/paper/2605.18054/) (2026)

## Abstract (arXiv)

Neural Radiance Field (NeRF) excels in photo-realistically static scenes, inspiring numerous efforts to facilitate volumetric videos. However, rendering dynamic and long-sequence radiance fields remains challenging due to the significant data required to represent volumetric videos. In this paper, we propose a novel end-to-end joint optimization scheme of dynamic NeRF representation and compression, called JointRF, thus achieving significantly improved quality and compression efficiency against the previous methods. Specifically, JointRF employs a compact residual feature grid and a coefficient feature grid to represent the dynamic NeRF. This representation handles large motions without compromising quality while concurrently diminishing temporal redundancy. We also introduce a sequential feature compression subnetwork to further reduce spatial-temporal redundancy. Finally, the representation and compression subnetworks are end-to-end trained combined within the JointRF. Extensive experiments demonstrate that JointRF can achieve superior compression performance across various datasets.