# FFNeRV: Flow-Guided Frame-Wise Neural Representations for Videos

arXiv: https://arxiv.org/abs/2212.12294 · HTML: https://mlsanity.com/paper/2212.12294/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 60 references that are in the map.
- First posted on arXiv 23 Dec 2022.

## Builds on (most cited first, 30 of 45)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2](https://mlsanity.com/paper/2112.14683/) (2021)
- [NeRV: Neural Representations for Videos](https://mlsanity.com/paper/2110.13903/) (2021)
- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [Editing Conditional Radiance Fields](https://mlsanity.com/paper/2105.06466/) (2021)
- [Implicit Neural Representations for Image Compression](https://mlsanity.com/paper/2112.04267/) (2021)
- [FVC: A New Framework towards Deep Video Compression in Feature Space](https://mlsanity.com/paper/2105.09600/) (2021)
- [Modulated Periodic Activations for Generalizable Local Functional Representations](https://mlsanity.com/paper/2104.03960/) (2021)

## Built on by (most cited first, 30 of 46)

- [HiNeRV: Video Compression with Hierarchical Encoding-based Neural Representation](https://mlsanity.com/paper/2306.09818/) (2023)
- [C3: High-performance and low-complexity neural compression from a single image or video](https://mlsanity.com/paper/2312.02753/) (2023)
- [USTC-TD: A Test Dataset and Benchmark for Image and Video Coding in 2020s](https://mlsanity.com/paper/2409.08481/) (2024)
- [Boosting Neural Representations for Videos with a Conditional Decoder](https://mlsanity.com/paper/2402.18152/) (2024)
- [NVRC: Neural Video Representation Compression](https://mlsanity.com/paper/2409.07414/) (2024)
- [DS-NeRV: Implicit Neural Video Representation with Decomposed Static and Dynamic Codes](https://mlsanity.com/paper/2403.15679/) (2024)
- [PNVC: Towards Practical INR-based Video Compression](https://mlsanity.com/paper/2409.00953/) (2024)
- [Continuous Memory Representation for Anomaly Detection](https://mlsanity.com/paper/2402.18293/) (2024)
- [Good, Cheap, and Fast: Overfitted Image Compression with Wasserstein Distortion](https://mlsanity.com/paper/2412.00505/) (2024)
- [Immersive Video Compression using Implicit Neural Representations](https://mlsanity.com/paper/2402.01596/) (2024)
- [Cool-chic video: Learned video coding with 800 parameters](https://mlsanity.com/paper/2402.03179/) (2024)
- [GIViC: Generative Implicit Video Compression](https://mlsanity.com/paper/2503.19604/) (2025)
- [SNeRV: Spectra-preserving Neural Representation for Video](https://mlsanity.com/paper/2501.01681/) (2025)
- [Implicit-explicit Integrated Representations for Multi-view Video Compression](https://mlsanity.com/paper/2311.17350/) (2023)
- [RT-NeRV: Rethinking Hybrid Neural Representations for Video via Residual Tokenization](https://mlsanity.com/paper/2403.12401/) (2024)
- [Progressively Deformable 2D Gaussian Splatting for Video Representation at Arbitrary Resolutions](https://mlsanity.com/paper/2503.05600/) (2025)
- [Coordinate-Aware Modulation for Neural Fields](https://mlsanity.com/paper/2311.14993/) (2023)
- [NERV++: An Enhanced Implicit Neural Video Representation](https://mlsanity.com/paper/2402.18305/) (2024)
- [MetaNeRV: Meta Neural Representations for Videos with Spatial-Temporal Guidance](https://mlsanity.com/paper/2501.02427/) (2025)
- [UAR-NVC: A Unified AutoRegressive Framework for Memory-Efficient Neural Video Compression](https://mlsanity.com/paper/2503.02733/) (2025)
- [How to Design and Train Your Implicit Neural Representation for Video Compression](https://mlsanity.com/paper/2506.24127/) (2025)
- [Enhanced Neural Video Representation Compression Across Extreme Complexity and Quality Scales](https://mlsanity.com/paper/2606.28163/) (2026)
- [Style Transfer to Calvin and Hobbes comics using Stable Diffusion](https://mlsanity.com/paper/2312.03993/) (2023)
- [High-Frequency Enhanced Hybrid Neural Representation for Video Compression](https://mlsanity.com/paper/2411.06685/) (2024)
- [Bias for Action: Video Implicit Neural Representations with Bias Modulation](https://mlsanity.com/paper/2501.09277/) (2025)
- [CANeRV: Content Adaptive Neural Representation for Video Compression](https://mlsanity.com/paper/2502.06181/) (2025)
- [Tree-NeRV: A Tree-Structured Neural Representation for Efficient Non-Uniform Video Encoding](https://mlsanity.com/paper/2504.12899/) (2025)
- [Efficient Neural Video Representation with Temporally Coherent Modulation](https://mlsanity.com/paper/2505.00335/) (2025)
- [SIEDD: Shared-Implicit Encoder with Discrete Decoders](https://mlsanity.com/paper/2506.23382/) (2025)
- [GSVR: 2D Gaussian-based Video Representation for 800+ FPS with Hybrid Deformation Field](https://mlsanity.com/paper/2507.05594/) (2025)

## Abstract (arXiv)

Neural fields, also known as coordinate-based or implicit neural representations, have shown a remarkable capability of representing, generating, and manipulating various forms of signals. For video representations, however, mapping pixel-wise coordinates to RGB colors has shown relatively low compression performance and slow convergence and inference speed. Frame-wise video representation, which maps a temporal coordinate to its entire frame, has recently emerged as an alternative method to represent videos, improving compression rates and encoding speed. While promising, it has still failed to reach the performance of state-of-the-art video compression algorithms. In this work, we propose FFNeRV, a novel method for incorporating flow information into frame-wise representations to exploit the temporal redundancy across the frames in videos inspired by the standard video codecs. Furthermore, we introduce a fully convolutional architecture, enabled by one-dimensional temporal grids, improving the continuity of spatial features. Experimental results show that FFNeRV yields the best performance for video compression and frame interpolation among the methods using frame-wise representations or neural fields. To reduce the model size even further, we devise a more compact convolutional architecture using the group and pointwise convolutions. With model compression techniques, including quantization-aware training and entropy coding, FFNeRV outperforms widely-used standard video codecs (H.264 and HEVC) and performs on par with state-of-the-art video compression algorithms.