# Perceptual Learned Image Compression via End-to-End JND-Based Optimization

arXiv: https://arxiv.org/abs/2402.02836 · HTML: https://mlsanity.com/paper/2402.02836/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 14 of 14)

- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding](https://mlsanity.com/paper/2203.10886/) (2022)
- [CompressAI: a PyTorch library and evaluation platform for end-to-end compression research](https://mlsanity.com/paper/2011.03029/) (2020)
- [Learned Image Compression with Mixed Transformer-CNN Architectures](https://mlsanity.com/paper/2303.14978/) (2023)
- [End-to-end optimization of nonlinear transform codes for perceptual quality](https://mlsanity.com/paper/1607.05006/) (2016)
- [VideoSet: A Large-Scale Compressed Video Quality Dataset Based on JND Measurement](https://mlsanity.com/paper/1701.01500/) (2017)
- [ProxIQA: A Proxy Approach to Perceptual Optimization of Learned Image Compression](https://mlsanity.com/paper/1910.08845/) (2019)
- [A Survey on Perceptually Optimized Video Coding](https://mlsanity.com/paper/2112.12284/) (2021)
- [Deep Perceptual Compression](https://mlsanity.com/paper/1907.08310/) (2019)
- [Self-Organized Variational Autoencoders (Self-VAE) for Learned Image Compression](https://mlsanity.com/paper/2105.12107/) (2021)
- [Comprehensive Complexity Assessment of Emerging Learned Image Compression on CPU and GPU](https://mlsanity.com/paper/2212.05466/) (2022)
- [JND-Based Perceptual Optimization For Learned Image Compression](https://mlsanity.com/paper/2302.13092/) (2023)
- [Perceptual impact of the loss function on deep-learning image coding performance](https://mlsanity.com/paper/2311.06084/) (2023)

## Built on by (most cited first, 1 of 1)

- [DT-JRD: Deep Transformer based Just Recognizable Difference Prediction Model for Video Coding for Machines](https://mlsanity.com/paper/2411.09308/) (2024)

## Abstract (arXiv)

Emerging Learned image Compression (LC) achieves significant improvements in coding efficiency by end-to-end training of neural networks for compression. An important benefit of this approach over traditional codecs is that any optimization criteria can be directly applied to the encoder-decoder networks during training. Perceptual optimization of LC to comply with the Human Visual System (HVS) is among such criteria, which has not been fully explored yet. This paper addresses this gap by proposing a novel framework to integrate Just Noticeable Distortion (JND) principles into LC. Leveraging existing JND datasets, three perceptual optimization methods are proposed to integrate JND into the LC training process: (1) Pixel-Wise JND Loss (PWL) prioritizes pixel-by-pixel fidelity in reproducing JND characteristics, (2) Image-Wise JND Loss (IWL) emphasizes on overall imperceptible degradation levels, and (3) Feature-Wise JND Loss (FWL) aligns the reconstructed image features with perceptually significant features. Experimental evaluations demonstrate the effectiveness of JND integration, highlighting improvements in rate-distortion performance and visual quality, compared to baseline methods. The proposed methods add no extra complexity after training.