# Modality-Agnostic Variational Compression of Implicit Neural Representations

arXiv: https://arxiv.org/abs/2301.09479 · HTML: https://mlsanity.com/paper/2301.09479/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 71 references that are in the map.
- First posted on arXiv 23 Jan 2023.

## Builds on (most cited first, 30 of 47)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [How Does Batch Normalization Help Optimization?](https://mlsanity.com/paper/1805.11604/) (2018)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [Lossy Image Compression with Compressive Autoencoders](https://mlsanity.com/paper/1703.00395/) (2017)
- [Learned Initializations for Optimizing Coordinate-Based Neural Representations](https://mlsanity.com/paper/2012.02189/) (2020)
- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [Generative Adversarial Networks for Extreme Learned Image Compression](https://mlsanity.com/paper/1804.02958/) (2018)
- [Understanding and Improving Layer Normalization](https://mlsanity.com/paper/1911.07013/) (2019)
- [Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks](https://mlsanity.com/paper/2202.10571/) (2022)
- [Implicit Neural Representations for Image Compression](https://mlsanity.com/paper/2112.04267/) (2021)
- [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018)
- [Modulated Periodic Activations for Generalizable Local Functional Representations](https://mlsanity.com/paper/2104.03960/) (2021)

## Built on by (most cited first, 22 of 22)

- [C3: High-performance and low-complexity neural compression from a single image or video](https://mlsanity.com/paper/2312.02753/) (2023)
- [Online Adaptation of Language Models with a Memory of Amortized Contexts](https://mlsanity.com/paper/2403.04317/) (2024)
- [Compression with Bayesian Implicit Neural Representations](https://mlsanity.com/paper/2305.19185/) (2023)
- [Efficient Learning With Sine-Activated Low-rank Matrices](https://mlsanity.com/paper/2403.19243/) (2024)
- [LEIA: Latent View-invariant Embeddings for Implicit 3D Articulation](https://mlsanity.com/paper/2409.06703/) (2024)
- [Nonparametric Teaching of Implicit Neural Representations](https://mlsanity.com/paper/2405.10531/) (2024)
- [Learning Large-scale Neural Fields via Context Pruned Meta-Learning](https://mlsanity.com/paper/2302.00617/) (2023)
- [Good, Cheap, and Fast: Overfitted Image Compression with Wasserstein Distortion](https://mlsanity.com/paper/2412.00505/) (2024)
- [Fast Training of Sinusoidal Neural Fields via Scaling Initialization](https://mlsanity.com/paper/2410.04779/) (2024)
- [RECOMBINER: Robust and Enhanced Compression with Bayesian Implicit Neural Representations](https://mlsanity.com/paper/2309.17182/) (2023)
- [Latent-INR: A Flexible Framework for Implicit Representations of Videos with Discriminative Semantics](https://mlsanity.com/paper/2408.02672/) (2024)
- [In Search of a Data Transformation That Accelerates Neural Field Training](https://mlsanity.com/paper/2311.17094/) (2023)
- [Improved Positional Encoding for Implicit Neural Representation based Compact Data Representation](https://mlsanity.com/paper/2311.06059/) (2023)
- [Accelerating Relative Entropy Coding with Space Partitioning](https://mlsanity.com/paper/2405.12203/) (2024)
- [Hybrid Neural Representations for Spherical Data](https://mlsanity.com/paper/2402.05965/) (2024)
- [Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated Experts](https://mlsanity.com/paper/2403.08477/) (2024)
- [UAR-NVC: A Unified AutoRegressive Framework for Memory-Efficient Neural Video Compression](https://mlsanity.com/paper/2503.02733/) (2025)
- [ARC: Anchored Representation Clouds for High-Resolution INR Classification](https://mlsanity.com/paper/2503.15156/) (2025)
- [NTK-Guided Implicit Neural Teaching](https://mlsanity.com/paper/2511.15487/) (2025)
- [Implicit Neural Representations for Simultaneous Reduction and Continuous Reconstruction of Multi-Altitude Climate Data](https://mlsanity.com/paper/2409.17367/) (2024)
- [D'OH: Decoder-Only Random Hypernetworks for Implicit Neural Representations](https://mlsanity.com/paper/2403.19163/) (2024)
- [Efficient Compression of Sparse Accelerator Data Using Implicit Neural Representations and Importance Sampling](https://mlsanity.com/paper/2412.01754/) (2024)

## Abstract (arXiv)

We introduce a modality-agnostic neural compression algorithm based on a functional view of data and parameterised as an Implicit Neural Representation (INR). Bridging the gap between latent coding and sparsity, we obtain compact latent representations non-linearly mapped to a soft gating mechanism. This allows the specialisation of a shared INR network to each data item through subnetwork selection. After obtaining a dataset of such latent representations, we directly optimise the rate/distortion trade-off in a modality-agnostic space using neural compression. Variational Compression of Implicit Neural Representations (VC-INR) shows improved performance given the same representational capacity pre quantisation while also outperforming previous quantisation schemes used for other INR techniques. Our experiments demonstrate strong results over a large set of diverse modalities using the same algorithm without any modality-specific inductive biases. We show results on images, climate data, 3D shapes and scenes as well as audio and video, introducing VC-INR as the first INR-based method to outperform codecs as well-known and diverse as JPEG 2000, MP3 and AVC/HEVC on their respective modalities.