# NIRVANA: Neural Implicit Representations of Videos with Adaptive Networks and Autoregressive Patch-wise Modeling

arXiv: https://arxiv.org/abs/2212.14593 · HTML: https://mlsanity.com/paper/2212.14593/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 53 references that are in the map.
- First posted on arXiv 30 Dec 2022.

## Builds on (most cited first, 30 of 35)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [The State of Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1902.09574/) (2019)
- [YouTube-8M: A Large-Scale Video Classification Benchmark](https://mlsanity.com/paper/1609.08675/) (2016)
- [Lossy Image Compression with Compressive Autoencoders](https://mlsanity.com/paper/1703.00395/) (2017)
- [NeRV: Neural Representations for Videos](https://mlsanity.com/paper/2110.13903/) (2021)
- [Learned Initializations for Optimizing Coordinate-Based Neural Representations](https://mlsanity.com/paper/2012.02189/) (2020)
- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [Implicit Neural Representations for Image Compression](https://mlsanity.com/paper/2112.04267/) (2021)
- [Towards Accurate Binary Convolutional Neural Network](https://mlsanity.com/paper/1711.11294/) (2017)
- [Training with Quantization Noise for Extreme Model Compression](https://mlsanity.com/paper/2004.07320/) (2020)
- [BACON: Band-limited Coordinate Networks for Multiscale Scene Representation](https://mlsanity.com/paper/2112.04645/) (2021)
- [Transformers as Meta-Learners for Implicit Neural Representations](https://mlsanity.com/paper/2208.02801/) (2022)
- [And the Bit Goes Down: Revisiting the Quantization of Neural Networks](https://mlsanity.com/paper/1907.05686/) (2019)
- [E-NeRV: Expedite Neural Video Representation with Disentangled Spatial-Temporal Context](https://mlsanity.com/paper/2207.08132/) (2022)
- [ACORN: Adaptive Coordinate Networks for Neural Scene Representation](https://mlsanity.com/paper/2105.02788/) (2021)
- [Implicit Neural Video Compression](https://mlsanity.com/paper/2112.11312/) (2021)
- [Meta-Learning Sparse Implicit Neural Representations](https://mlsanity.com/paper/2110.14678/) (2021)
- [The Lottery Ticket Hypothesis for Object Recognition](https://mlsanity.com/paper/2012.04643/) (2020)
- [On the Out-of-distribution Generalization of Probabilistic Image Modelling](https://mlsanity.com/paper/2109.02639/) (2021)
- [PS-NeRV: Patch-wise Stylized Neural Representations for Videos](https://mlsanity.com/paper/2208.03742/) (2022)
- [Scalable Model Compression by Entropy Penalized Reparameterization](https://mlsanity.com/paper/1906.06624/) (2019)
- [Scalable Neural Video Representations with Learnable Positional Features](https://mlsanity.com/paper/2210.06823/) (2022)

## Built on by (most cited first, 30 of 51)

- [EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS](https://mlsanity.com/paper/2312.04564/) (2023)
- [HNeRV: A Hybrid Neural Representation for Videos](https://mlsanity.com/paper/2304.02633/) (2023)
- [C3: High-performance and low-complexity neural compression from a single image or video](https://mlsanity.com/paper/2312.02753/) (2023)
- [Towards Scalable Neural Representation for Diverse Videos](https://mlsanity.com/paper/2303.14124/) (2023)
- [SHACIRA: Scalable HAsh-grid Compression for Implicit Neural Representations](https://mlsanity.com/paper/2309.15848/) (2023)
- [INCODE: Implicit Neural Conditioning with Prior Knowledge Embeddings](https://mlsanity.com/paper/2310.18846/) (2023)
- [Boosting Neural Representations for Videos with a Conditional Decoder](https://mlsanity.com/paper/2402.18152/) (2024)
- [NVRC: Neural Video Representation Compression](https://mlsanity.com/paper/2409.07414/) (2024)
- [DS-NeRV: Implicit Neural Video Representation with Decomposed Static and Dynamic Codes](https://mlsanity.com/paper/2403.15679/) (2024)
- [Learning Transferable Features for Implicit Neural Representations](https://mlsanity.com/paper/2409.09566/) (2024)
- [Bidirectional Autoregressive Diffusion Model for Dance Generation](https://mlsanity.com/paper/2402.04356/) (2024)
- [Good, Cheap, and Fast: Overfitted Image Compression with Wasserstein Distortion](https://mlsanity.com/paper/2412.00505/) (2024)
- [Few-shot Novel View Synthesis using Depth Aware 3D Gaussian Splatting](https://mlsanity.com/paper/2410.11080/) (2024)
- [Latent-INR: A Flexible Framework for Implicit Representations of Videos with Discriminative Semantics](https://mlsanity.com/paper/2408.02672/) (2024)
- [SNeRV: Spectra-preserving Neural Representation for Video](https://mlsanity.com/paper/2501.01681/) (2025)
- [Implicit-explicit Integrated Representations for Multi-view Video Compression](https://mlsanity.com/paper/2311.17350/) (2023)
- [EVOS: Efficient Implicit Neural Training via EVOlutionary Selector](https://mlsanity.com/paper/2412.10153/) (2024)
- [Continual Learning: Forget-free Winning Subnetworks for Video Representations](https://mlsanity.com/paper/2312.11973/) (2023)
- [FCNR: Fast Compressive Neural Representation of Visualization Images](https://mlsanity.com/paper/2407.16369/) (2024)
- [Explaining the Implicit Neural Canvas: Connecting Pixels to Neurons by Tracing their Contributions](https://mlsanity.com/paper/2401.10217/) (2024)
- [Ultra-lightweight Neural Video Representation Compression](https://mlsanity.com/paper/2512.04019/) (2025)
- [Progressive Fourier Neural Representation for Sequential Video Compilation](https://mlsanity.com/paper/2306.11305/) (2023)
- [NERV++: An Enhanced Implicit Neural Video Representation](https://mlsanity.com/paper/2402.18305/) (2024)
- [INRetouch: Context Aware Implicit Neural Representation for Photography Retouching](https://mlsanity.com/paper/2412.03848/) (2024)
- [UAR-NVC: A Unified AutoRegressive Framework for Memory-Efficient Neural Video Compression](https://mlsanity.com/paper/2503.02733/) (2025)
- [How to Design and Train Your Implicit Neural Representation for Video Compression](https://mlsanity.com/paper/2506.24127/) (2025)
- [Enhanced Neural Video Representation Compression Across Extreme Complexity and Quality Scales](https://mlsanity.com/paper/2606.28163/) (2026)
- [Predicting the Encoding Error of SIRENs](https://mlsanity.com/paper/2410.21645/) (2024)
- [Bias for Action: Video Implicit Neural Representations with Bias Modulation](https://mlsanity.com/paper/2501.09277/) (2025)
- [CANeRV: Content Adaptive Neural Representation for Video Compression](https://mlsanity.com/paper/2502.06181/) (2025)

## Abstract (arXiv)

Implicit Neural Representations (INR) have recently shown to be powerful tool for high-quality video compression. However, existing works are limiting as they do not explicitly exploit the temporal redundancy in videos, leading to a long encoding time. Additionally, these methods have fixed architectures which do not scale to longer videos or higher resolutions. To address these issues, we propose NIRVANA, which treats videos as groups of frames and fits separate networks to each group performing patch-wise prediction. This design shares computation within each group, in the spatial and temporal dimensions, resulting in reduced encoding time of the video. The video representation is modeled autoregressively, with networks fit on a current group initialized using weights from the previous group's model. To further enhance efficiency, we perform quantization of the network parameters during training, requiring no post-hoc pruning or quantization. When compared with previous works on the benchmark UVG dataset, NIRVANA improves encoding quality from 37.36 to 37.70 (in terms of PSNR) and the encoding speed by 12X, while maintaining the same compression rate. In contrast to prior video INR works which struggle with larger resolution and longer videos, we show that our algorithm is highly flexible and scales naturally due to its patch-wise and autoregressive designs. Moreover, our method achieves variable bitrate compression by adapting to videos with varying inter-frame motion. NIRVANA achieves 6X decoding speed and scales well with more GPUs, making it practical for various deployment scenarios.