# Learning sparse auto-encoders for green AI image coding

arXiv: https://arxiv.org/abs/2209.04448 · HTML: https://mlsanity.com/paper/2209.04448/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 9 Sep 2022.

## Builds on (most cited first, 16 of 16)

- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [High-Fidelity Generative Image Compression](https://mlsanity.com/paper/2006.09965/) (2020)
- [Lossy Image Compression with Compressive Autoencoders](https://mlsanity.com/paper/1703.00395/) (2017)
- [CompressAI: a PyTorch library and evaluation platform for end-to-end compression research](https://mlsanity.com/paper/2011.03029/) (2020)
- [Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask](https://mlsanity.com/paper/1905.01067/) (2019)
- [The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink](https://mlsanity.com/paper/2204.05149/) (2022)
- [Data-Driven Sparse Structure Selection for Deep Neural Networks](https://mlsanity.com/paper/1707.01213/) (2017)
- [Learning the Number of Neurons in Deep Networks](https://mlsanity.com/paper/1611.06321/) (2016)
- [Deep Convolutional AutoEncoder-based Lossy Image Compression](https://mlsanity.com/paper/1804.09535/) (2018)
- [Computationally Efficient Neural Image Compression](https://mlsanity.com/paper/1912.08771/) (2019)
- [Learning Sparse Neural Networks via Sensitivity-Driven Regularization](https://mlsanity.com/paper/1810.11764/) (2018)
- [Classification and regression using an outer approximation projection-gradient method](https://mlsanity.com/paper/1506.02196/) (2015)

## Built on by (most cited first, 3 of 3)

- [Neural Image Compression: Generalization, Robustness, and Spectral Biases](https://mlsanity.com/paper/2307.08657/) (2023)
- [Near-Linear Time Projection onto the ℓ1,∞ Ball; Application to Sparse Autoencoders](https://mlsanity.com/paper/2307.09836/) (2023)
- [A new Linear Time Bi-level ℓ1,∞ projection ; Application to the sparsification of auto-encoders neural networks](https://mlsanity.com/paper/2407.16293/) (2024)

## Abstract (arXiv)

Recently, convolutional auto-encoders (CAE) were introduced for image coding. They achieved performance improvements over the state-of-the-art JPEG2000 method. However, these performances were obtained using massive CAEs featuring a large number of parameters and whose training required heavy computational power. In this paper, we address the problem of lossy image compression using a CAE with a small memory footprint and low computational power usage. In order to overcome the computational cost issue, the majority of the literature uses Lagrangian proximal regularization methods, which are time consuming themselves. In this work, we propose a constrained approach and a new structured sparse learning method. We design an algorithm and test it on three constraints: the classical ℓ1 constraint, the ℓ1,∞ and the new ℓ1,1 constraint. Experimental results show that the ℓ1,1 constraint provides the best structured sparsity, resulting in a high reduction of memory and computational cost, with similar rate-distortion performance as with dense networks.