# TinyNeRV: Compact Neural Video Representations via Capacity Scaling, Distillation, and Low-Precision Inference

arXiv: https://arxiv.org/abs/2604.09220 · HTML: https://mlsanity.com/paper/2604.09220/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 43 of its 85 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Builds on (most cited first, 30 of 43)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Explaining Neural Scaling Laws](https://mlsanity.com/paper/2102.06701/) (2021)
- [Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks](https://mlsanity.com/paper/2004.05937/) (2020)
- [NeRV: Neural Representations for Videos](https://mlsanity.com/paper/2110.13903/) (2021)
- [Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets](https://mlsanity.com/paper/1903.05662/) (2019)
- [Quantization Networks](https://mlsanity.com/paper/1911.09464/) (2019)
- [Overcoming Oscillations in Quantization-Aware Training](https://mlsanity.com/paper/2203.11086/) (2022)
- [HNeRV: A Hybrid Neural Representation for Videos](https://mlsanity.com/paper/2304.02633/) (2023)
- [Towards Practical Real-Time Neural Video Compression](https://mlsanity.com/paper/2502.20762/) (2025)
- [E-NeRV: Expedite Neural Video Representation with Disentangled Spatial-Temporal Context](https://mlsanity.com/paper/2207.08132/) (2022)
- [HiNeRV: Video Compression with Hierarchical Encoding-based Neural Representation](https://mlsanity.com/paper/2306.09818/) (2023)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [Frequency-Aligned Knowledge Distillation for Lightweight Spatiotemporal Forecasting](https://mlsanity.com/paper/2507.02939/) (2025)
- [A Comprehensive Survey on Knowledge Distillation](https://mlsanity.com/paper/2503.12067/) (2025)
- [FFNeRV: Flow-Guided Frame-Wise Neural Representations for Videos](https://mlsanity.com/paper/2212.12294/) (2022)
- [DNeRV: Modeling Inherent Dynamics via Difference Neural Representation for Videos](https://mlsanity.com/paper/2304.06544/) (2023)
- [Lightweight Deep Learning for Resource-Constrained Environments: A Survey](https://mlsanity.com/paper/2404.07236/) (2024)
- [Boosting Neural Representations for Videos with a Conditional Decoder](https://mlsanity.com/paper/2402.18152/) (2024)
- [Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning](https://mlsanity.com/paper/2308.07209/) (2023)
- [From Zero to Detail: Deconstructing Ultra-High-Definition Image Restoration from Progressive Spectral Perspective](https://mlsanity.com/paper/2503.13165/) (2025)
- [Resource-Efficient Neural Networks for Embedded Systems](https://mlsanity.com/paper/2001.03048/) (2020)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [LittleBit: Ultra Low-Bit Quantization via Latent Factorization](https://mlsanity.com/paper/2506.13771/) (2025)
- [Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review](https://mlsanity.com/paper/2311.11883/) (2023)
- [SNeRV: Spectra-preserving Neural Representation for Video](https://mlsanity.com/paper/2501.01681/) (2025)
- [Applications of Knowledge Distillation in Remote Sensing: A Survey](https://mlsanity.com/paper/2409.12111/) (2024)

## Abstract (arXiv)

Implicit neural video representations encode entire video sequences within the parameters of a neural network and enable constant time frame reconstruction. Recent work on Neural Representations for Videos (NeRV) has demonstrated competitive reconstruction performance while avoiding the sequential decoding process of conventional video codecs. However, most existing studies focus on moderate or high capacity models, leaving the behavior of extremely compact configurations required for constrained environments insufficiently explored. This paper presents a systematic study of tiny NeRV architectures designed for efficient deployment. Two lightweight configurations, NeRV-T and NeRV-T+, are introduced and evaluated across multiple video datasets in order to analyze how aggressive capacity reduction affects reconstruction quality, computational complexity, and decoding throughput. Beyond architectural scaling, the work investigates strategies for improving the performance of compact models without increasing inference cost. Knowledge distillation with frequency-aware focal supervision is explored to enhance reconstruction fidelity in low-capacity networks. In addition, the impact of lowprecision inference is examined through both post training quantization and quantization aware training to study the robustness of tiny models under reduced numerical precision. Experimental results demonstrate that carefully designed tiny NeRV variants can achieve favorable quality efficiency trade offs while substantially reducing parameter count, computational cost, and memory requirements. These findings provide insight into the practical limits of compact neural video representations and offer guidance for deploying NeRV style models in resource constrained and real-time environments. The official implementation is available at https: //github.com/HannanAkhtar/TinyNeRV-Implementation.