# DS-NeRV: Implicit Neural Video Representation with Decomposed Static and Dynamic Codes

arXiv: https://arxiv.org/abs/2403.15679 · HTML: https://mlsanity.com/paper/2403.15679/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 55 references that are in the map.
- First posted on arXiv 23 Mar 2024.

## Builds on (most cited first, 30 of 39)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video](https://mlsanity.com/paper/2012.12247/) (2020)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [Neural Radiance Flow for 4D View Synthesis and Video Processing](https://mlsanity.com/paper/2012.09790/) (2020)
- [Cross-view Transformers for real-time Map-view Semantic Segmentation](https://mlsanity.com/paper/2205.02833/) (2022)
- [NeRV: Neural Representations for Videos](https://mlsanity.com/paper/2110.13903/) (2021)
- [NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior](https://mlsanity.com/paper/2212.07388/) (2022)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models](https://mlsanity.com/paper/2208.06677/) (2022)
- [Optimizing the Latent Space of Generative Networks](https://mlsanity.com/paper/1707.05776/) (2017)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [Video Frame Interpolation Transformer](https://mlsanity.com/paper/2111.13817/) (2021)
- [M-LVC: Multiple Frames Prediction for Learned Video Compression](https://mlsanity.com/paper/2004.10290/) (2020)
- [HNeRV: A Hybrid Neural Representation for Videos](https://mlsanity.com/paper/2304.02633/) (2023)
- [Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation](https://mlsanity.com/paper/2303.00440/) (2023)
- [Learning for Video Compression with Hierarchical Quality and Recurrent Enhancement](https://mlsanity.com/paper/2003.01966/) (2020)
- [E-NeRV: Expedite Neural Video Representation with Disentangled Spatial-Temporal Context](https://mlsanity.com/paper/2207.08132/) (2022)
- [ELF-VC: Efficient Learned Flexible-Rate Video Coding](https://mlsanity.com/paper/2104.14335/) (2021)
- [Few-View Object Reconstruction with Unknown Categories and Camera Poses](https://mlsanity.com/paper/2212.04492/) (2022)

## Built on by (most cited first, 25 of 25)

- [GaussianVideo: Efficient Video Representation via Hierarchical Gaussian Splatting](https://mlsanity.com/paper/2501.04782/) (2025)
- [GIViC: Generative Implicit Video Compression](https://mlsanity.com/paper/2503.19604/) (2025)
- [AdaGaR: Adaptive Gabor Representation for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2601.00796/) (2026)
- [Progressively Deformable 2D Gaussian Splatting for Video Representation at Arbitrary Resolutions](https://mlsanity.com/paper/2503.05600/) (2025)
- [Continuous Representation Methods, Theories, and Applications: An Overview and Perspectives](https://mlsanity.com/paper/2505.15222/) (2025)
- [Neural Video Representation for Redundancy Reduction and Consistency Preservation](https://mlsanity.com/paper/2409.18497/) (2024)
- [UAR-NVC: A Unified AutoRegressive Framework for Memory-Efficient Neural Video Compression](https://mlsanity.com/paper/2503.02733/) (2025)
- [How to Design and Train Your Implicit Neural Representation for Video Compression](https://mlsanity.com/paper/2506.24127/) (2025)
- [Bias for Action: Video Implicit Neural Representations with Bias Modulation](https://mlsanity.com/paper/2501.09277/) (2025)
- [Tree-NeRV: A Tree-Structured Neural Representation for Efficient Non-Uniform Video Encoding](https://mlsanity.com/paper/2504.12899/) (2025)
- [SR-NeRV: Improving Embedding Efficiency of Neural Video Representation via Super-Resolution](https://mlsanity.com/paper/2505.00046/) (2025)
- [SIEDD: Shared-Implicit Encoder with Discrete Decoders](https://mlsanity.com/paper/2506.23382/) (2025)
- [GSVR: 2D Gaussian-based Video Representation for 800+ FPS with Hybrid Deformation Field](https://mlsanity.com/paper/2507.05594/) (2025)
- [STGV: Spatio-Temporal Hash Encoding for Gaussian-based Video Representation](https://mlsanity.com/paper/2604.10910/) (2026)
- [Range Image-Based Implicit Neural Compression for LiDAR Point Clouds](https://mlsanity.com/paper/2504.17229/) (2025)
- [Implicit Neural Representation Facilitates Unified Universal Vision Encoding](https://mlsanity.com/paper/2601.14256/) (2026)
- [TeCoNeRV: Leveraging Temporal Coherence for Compressible Neural Representations for Videos](https://mlsanity.com/paper/2602.16711/) (2026)
- [CWRNN-INVR: A Coupled WarpRNN based Implicit Neural Video Representation](https://mlsanity.com/paper/2604.06564/) (2026)
- [Structure-Preserving Patch Decoding for Efficient Neural Video Representation](https://mlsanity.com/paper/2506.12896/) (2025)
- [T-MLP: Tailed Multi-Layer Perceptron for Level-of-Detail Signal Representation](https://mlsanity.com/paper/2509.00066/) (2025)
- [LRConv-NeRV: Low Rank Convolution for Efficient Neural Video Compression](https://mlsanity.com/paper/2603.18261/) (2026)
- [NeR-SC: Adapting Neural Video Representation to Screen Content](https://mlsanity.com/paper/2605.27024/) (2026)
- [Video Compression with Graph-inspired Neural Representation](https://mlsanity.com/paper/2609.05541/) (2026)
- [Resolution-Flexible Decoding for Hybrid Neural Video Representations](https://mlsanity.com/paper/2609.23555/) (2026)
- [Affine-Aligned Atlas for Canonical Gaussian Construction in Video Representation](https://mlsanity.com/paper/2610.01114/) (2026)

## Abstract (arXiv)

Implicit neural representations for video (NeRV) have recently become a novel way for high-quality video representation. However, existing works employ a single network to represent the entire video, which implicitly confuse static and dynamic information. This leads to an inability to effectively compress the redundant static information and lack the explicitly modeling of global temporal-coherent dynamic details. To solve above problems, we propose DS-NeRV, which decomposes videos into sparse learnable static codes and dynamic codes without the need for explicit optical flow or residual supervision. By setting different sampling rates for two codes and applying weighted sum and interpolation sampling methods, DS-NeRV efficiently utilizes redundant static information while maintaining high-frequency details. Additionally, we design a cross-channel attention-based (CCA) fusion module to efficiently fuse these two codes for frame decoding. Our approach achieves a high quality reconstruction of 31.2 PSNR with only 0.35M parameters thanks to separate static and dynamic codes representation and outperforms existing NeRV methods in many downstream tasks. Our project website is at https://haoyan14.github.io/DS-NeRV.