mlsanityData to 14 Oct 2026

Efficient Neural Video Representation with Temporally Coherent Modulation

arXiv 2505.00335 ↗First posted 1 May 2025cs.CV cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Implicit neural representations (INR) has found successful applications across diverse domains. To employ INR in real-life, it is important to speed up training. In the field of INR for video applications, the state-of-the-art approach employs grid-type parametric encoding and successfully achieves a faster encoding speed in comparison to its predecessors. However, the grid usage, which does not consider the video's dynamic nature, leads to redundant use of trainable parameters. As a result, it has significantly lower parameter efficiency and higher bitrate compared to NeRV-style methods that do not use a parametric encoding. To address the problem, we propose Neural Video representation with Temporally coherent Modulation (NVTM), a novel framework that can capture dynamic characteristics of video. By decomposing the spatio-temporal 3D video data into a set of 2D grids with flow information, NVTM enables learning video representation rapidly and uses parameter efficiently. Our framework enables to process temporally corresponding pixels at once, resulting in the fastest encoding speed for a reasonable video quality, especially when compared to the NeRV-style method, with a speed increase of over 3 times. Also, it remarks an average of 1.54dB/0.019 improvements in PSNR/LPIPS on UVG (Dynamic) (even with 10% fewer parameters) and an average of 1.84dB/0.013 improvements in PSNR/LPIPS on MCL-JCV (Dynamic), compared to previous grid-type works. By expanding this to compression tasks, we demonstrate comparable performance to video compression standards (H.264, HEVC) and recent INR approaches for video compression. Additionally, we perform extensive experiments demonstrating the superior performance of our algorithm across diverse tasks, encompassing super resolution, frame interpolation and video inpainting. Project page is https://sujiikim.github.io/NVTM/.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 33 of its 48 references that are in the map. First posted on arXiv 1 May 2025.

Builds on

33 of 48 references.

References · 33

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  4. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  5. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
  6. 2020Neural Sparse Voxel Fieldscited by 959
  7. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  8. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 552
  9. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  10. 2020Local Implicit Grid Representations for 3D Scenescited by 373
  11. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  12. 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
21 more
  1. 2021NeRV: Neural Representations for Videoscited by 256
  2. 2021COIN: COmpression with Implicit Neural representationscited by 233
  3. 2022From data to functa: Your data point is a function and you can treat it like onecited by 187
  4. 2021Modulated Periodic Activations for Generalizable Local Functional Representationscited by 140
  5. 2022VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolutioncited by 101
  6. 2023HNeRV: A Hybrid Neural Representation for Videoscited by 100
  7. 2022COIN++: Neural Compression Across Modalitiescited by 82
  8. 2022E-NeRV: Expedite Neural Video Representation with Disentangled Spatial-Temporal Contextcited by 76
  9. 2023Spatial Functa: Scaling Functa to ImageNet Classification and Generationcited by 57
  10. 2022NIRVANA: Neural Implicit Representations of Videos with Adaptive Networks and Autoregressive Patch-wise Modelingcited by 51
  11. 2022Compressing Explicit Voxel Grid Representations: fast NeRFs become also smallcited by 48
  12. 2022PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequencecited by 46
  13. 2022FFNeRV: Flow-Guided Frame-Wise Neural Representations for Videoscited by 46
  14. 2023DNeRV: Modeling Inherent Dynamics via Difference Neural Representation for Videoscited by 42
  15. 2023Towards Scalable Neural Representation for Diverse Videoscited by 37
  16. 2022PS-NeRV: Patch-wise Stylized Neural Representations for Videoscited by 32
  17. 2023SHACIRA: Scalable HAsh-grid Compression for Implicit Neural Representationscited by 31
  18. 2022Scalable Neural Video Representations with Learnable Positional Featurescited by 29
  19. 2022One is All: Bridging the Gap Between Neural Radiance Fields Architectures with Progressive Volume Distillationcited by 18
  20. 2022Neural Residual Flow Fields for Efficient Video Representationscited by 16
  21. 2022Frame Interpolation for Dynamic Scenes with Implicit Flow Encodingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026AdaGaR: Adaptive Gabor Representation for Dynamic Scene Reconstructioncited by 9
  2. 2026TeCoNeRV: Leveraging Temporal Coherence for Compressible Neural Representations for Videoscited by 1