# Self-Organized Variational Autoencoders (Self-VAE) for Learned Image Compression

arXiv: https://arxiv.org/abs/2105.12107 · HTML: https://mlsanity.com/paper/2105.12107/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 25 May 2021.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [Lossy Image Compression with Compressive Autoencoders](https://mlsanity.com/paper/1703.00395/) (2017)
- [Channel-wise Autoregressive Entropy Models for Learned Image Compression](https://mlsanity.com/paper/2007.08739/) (2020)
- [Nonlinear Transform Coding](https://mlsanity.com/paper/2007.03034/) (2020)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [End-to-end optimization of nonlinear transform codes for perceptual quality](https://mlsanity.com/paper/1607.05006/) (2016)
- [Self-Organized Operational Neural Networks with Generative Neurons](https://mlsanity.com/paper/2004.11778/) (2020)
- [Operational Neural Networks](https://mlsanity.com/paper/1902.11106/) (2019)
- [Self-Organized Operational Neural Networks for Severe Image Restoration Problems](https://mlsanity.com/paper/2008.12894/) (2020)
- [FastONN -- Python based open-source GPU implementation for Operational Neural Networks](https://mlsanity.com/paper/2006.02267/) (2020)
- [Heterogeneous Multilayer Generalized Operational Perceptron](https://mlsanity.com/paper/1804.05093/) (2018)
- [Progressive Operational Perceptron with Memory](https://mlsanity.com/paper/1808.06377/) (2018)

## Built on by (most cited first, 8 of 8)

- [OSegNet: Operational Segmentation Network for COVID-19 Detection using Chest X-ray Images](https://mlsanity.com/paper/2202.10185/) (2022)
- [Advancements in Generative AI: A Comprehensive Review of GANs, GPT, Autoencoders, Diffusion Model, and Transformers](https://mlsanity.com/paper/2311.10242/) (2023)
- [Clinically Relevant Latent Space Embedding of Cancer Histopathology Slides through Variational Autoencoder Based Image Compression](https://mlsanity.com/paper/2303.13332/) (2023)
- [End-to-End Learned Block-Based Image Compression with Block-Level Masked Convolutions and Asymptotic Closed Loop Training](https://mlsanity.com/paper/2203.11686/) (2022)
- [Channel-wise Feature Decorrelation for Enhanced Learned Image Compression](https://mlsanity.com/paper/2403.10936/) (2024)
- [PAON: A New Neuron Model using Pad'e Approximants](https://mlsanity.com/paper/2403.11791/) (2024)
- [Perceptual Learned Image Compression via End-to-End JND-Based Optimization](https://mlsanity.com/paper/2402.02836/) (2024)
- [Pad'e Neurons for Efficient Neural Models](https://mlsanity.com/paper/2601.04005/) (2026)

## Abstract (arXiv)

In end-to-end optimized learned image compression, it is standard practice to use a convolutional variational autoencoder with generalized divisive normalization (GDN) to transform images into a latent space. Recently, Operational Neural Networks (ONNs) that learn the best non-linearity from a set of alternatives, and their self-organized variants, Self-ONNs, that approximate any non-linearity via Taylor series have been proposed to address the limitations of convolutional layers and a fixed nonlinear activation. In this paper, we propose to replace the convolutional and GDN layers in the variational autoencoder with self-organized operational layers, and propose a novel self-organized variational autoencoder (Self-VAE) architecture that benefits from stronger non-linearity. The experimental results demonstrate that the proposed Self-VAE yields improvements in both rate-distortion performance and perceptual image quality.