# HiNeRV: Video Compression with Hierarchical Encoding-based Neural Representation

arXiv: https://arxiv.org/abs/2306.09818 · HTML: https://mlsanity.com/paper/2306.09818/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 62 references that are in the map.
- First posted on arXiv 16 Jun 2023.

## Builds on (most cited first, 30 of 43)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [NeRV: Neural Representations for Videos](https://mlsanity.com/paper/2110.13903/) (2021)
- [Learned Initializations for Optimizing Coordinate-Based Neural Representations](https://mlsanity.com/paper/2012.02189/) (2020)
- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [Neural Video Compression with Diverse Contexts](https://mlsanity.com/paper/2302.14402/) (2023)
- [FVC: A New Framework towards Deep Video Compression in Feature Space](https://mlsanity.com/paper/2105.09600/) (2021)
- [Training with Quantization Noise for Extreme Model Compression](https://mlsanity.com/paper/2004.07320/) (2020)
- [Modulated Periodic Activations for Generalizable Local Functional Representations](https://mlsanity.com/paper/2104.03960/) (2021)
- [Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compression](https://mlsanity.com/paper/2207.05894/) (2022)
- [Temporal Context Mining for Learned Video Compression](https://mlsanity.com/paper/2111.13850/) (2021)
- [VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution](https://mlsanity.com/paper/2206.04647/) (2022)
- [HNeRV: A Hybrid Neural Representation for Videos](https://mlsanity.com/paper/2304.02633/) (2023)

## Built on by (most cited first, 30 of 71)

- [C3: High-performance and low-complexity neural compression from a single image or video](https://mlsanity.com/paper/2312.02753/) (2023)
- [USTC-TD: A Test Dataset and Benchmark for Image and Video Coding in 2020s](https://mlsanity.com/paper/2409.08481/) (2024)
- [Boosting Neural Representations for Videos with a Conditional Decoder](https://mlsanity.com/paper/2402.18152/) (2024)
- [NVRC: Neural Video Representation Compression](https://mlsanity.com/paper/2409.07414/) (2024)
- [PNVC: Towards Practical INR-based Video Compression](https://mlsanity.com/paper/2409.00953/) (2024)
- [HIIF: Hierarchical Encoding based Implicit Image Function for Continuous Super-resolution](https://mlsanity.com/paper/2412.03748/) (2024)
- [Neural Video Compression with Context Modulation](https://mlsanity.com/paper/2505.14541/) (2025)
- [Good, Cheap, and Fast: Overfitted Image Compression with Wasserstein Distortion](https://mlsanity.com/paper/2412.00505/) (2024)
- [Immersive Video Compression using Implicit Neural Representations](https://mlsanity.com/paper/2402.01596/) (2024)
- [Cool-chic video: Learned video coding with 800 parameters](https://mlsanity.com/paper/2402.03179/) (2024)
- [GIViC: Generative Implicit Video Compression](https://mlsanity.com/paper/2503.19604/) (2025)
- [SNeRV: Spectra-preserving Neural Representation for Video](https://mlsanity.com/paper/2501.01681/) (2025)
- [Compressing Deep Image Super-resolution Models](https://mlsanity.com/paper/2401.00523/) (2023)
- [RT-NeRV: Rethinking Hybrid Neural Representations for Video via Residual Tokenization](https://mlsanity.com/paper/2403.12401/) (2024)
- [GSVC: Efficient Video Representation and Compression Through 2D Gaussian Splatting](https://mlsanity.com/paper/2501.12060/) (2025)
- [FCNR: Fast Compressive Neural Representation of Visualization Images](https://mlsanity.com/paper/2407.16369/) (2024)
- [Benchmarking Conventional and Learned Video Codecs with a Low-Delay Configuration](https://mlsanity.com/paper/2408.05042/) (2024)
- [C2D-ISR: Optimizing Attention-based Image Super-resolution from Continuous to Discrete Scales](https://mlsanity.com/paper/2503.13740/) (2025)
- [Ultra-Fast Neural Video Compression](https://mlsanity.com/paper/2606.04410/) (2026)
- [Prediction and Reference Quality Adaptation for Learned Video Compression](https://mlsanity.com/paper/2406.14118/) (2024)
- [RTSR: A Real-Time Super-Resolution Model for AV1 Compressed Content](https://mlsanity.com/paper/2411.13362/) (2024)
- [Progressively Deformable 2D Gaussian Splatting for Video Representation at Arbitrary Resolutions](https://mlsanity.com/paper/2503.05600/) (2025)
- [Ultra-lightweight Neural Video Representation Compression](https://mlsanity.com/paper/2512.04019/) (2025)
- [Accelerating Learnt Video Codecs with Gradient Decay and Layer-wise Distillation](https://mlsanity.com/paper/2312.02605/) (2023)
- [SPC-NeRF: Spatial Predictive Compression for Voxel Based Radiance Field](https://mlsanity.com/paper/2402.16366/) (2024)
- [Blind Video Super-Resolution based on Implicit Kernels](https://mlsanity.com/paper/2503.07856/) (2025)
- [NERV++: An Enhanced Implicit Neural Video Representation](https://mlsanity.com/paper/2402.18305/) (2024)
- [ReALLM: A general framework for LLM compression and fine-tuning](https://mlsanity.com/paper/2405.13155/) (2024)
- [NVC-1B: A Large Neural Video Coding Model](https://mlsanity.com/paper/2407.19402/) (2024)
- [MetaNeRV: Meta Neural Representations for Videos with Spatial-Temporal Guidance](https://mlsanity.com/paper/2501.02427/) (2025)

## Abstract (arXiv)

Learning-based video compression is currently a popular research topic, offering the potential to compete with conventional standard video codecs. In this context, Implicit Neural Representations (INRs) have previously been used to represent and compress image and video content, demonstrating relatively high decoding speed compared to other methods. However, existing INR-based methods have failed to deliver rate quality performance comparable with the state of the art in video compression. This is mainly due to the simplicity of the employed network architectures, which limit their representation capability. In this paper, we propose HiNeRV, an INR that combines light weight layers with novel hierarchical positional encodings. We employs depth-wise convolutional, MLP and interpolation layers to build the deep and wide network architecture with high capacity. HiNeRV is also a unified representation encoding videos in both frames and patches at the same time, which offers higher performance and flexibility than existing methods. We further build a video codec based on HiNeRV and a refined pipeline for training, pruning and quantization that can better preserve HiNeRV's performance during lossy model compression. The proposed method has been evaluated on both UVG and MCL-JCV datasets for video compression, demonstrating significant improvement over all existing INRs baselines and competitive performance when compared to learning-based codecs (72.3% overall bit rate saving over HNeRV and 43.4% over DCVC on the UVG dataset, measured in PSNR).