# Viewport-based Neural 360deg Image Compression

arXiv: https://arxiv.org/abs/2603.22776 · HTML: https://mlsanity.com/paper/2603.22776/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 55 references that are in the map.
- First posted on arXiv 24 Mar 2026.

## Builds on (most cited first, 25 of 25)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [CompressAI: a PyTorch library and evaluation platform for end-to-end compression research](https://mlsanity.com/paper/2011.03029/) (2020)
- [Generative Adversarial Networks for Extreme Learned Image Compression](https://mlsanity.com/paper/1804.02958/) (2018)
- [Checkerboard Context Model for Efficient Learned Image Compression](https://mlsanity.com/paper/2103.15306/) (2021)
- [TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up](https://mlsanity.com/paper/2102.07074/) (2021)
- [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018)
- [Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks](https://mlsanity.com/paper/1703.10114/) (2017)
- [Transformer-based Image Compression](https://mlsanity.com/paper/2111.06707/) (2021)
- [Learning Accurate Entropy Model with Global Reference for Image Compression](https://mlsanity.com/paper/2010.08321/) (2020)
- [Contextformer: A Transformer with Spatio-Channel Attention for Context Modeling in Learned Image Compression](https://mlsanity.com/paper/2203.02452/) (2022)
- [Learned Video Compression via Joint Spatial-Temporal Correlation Exploration](https://mlsanity.com/paper/1912.06348/) (2019)
- [Quantum Neural Network Compression](https://mlsanity.com/paper/2207.01578/) (2022)
- [Unified Static and Dynamic Network: Efficient Temporal Filtering for Video Grounding](https://mlsanity.com/paper/2403.14174/) (2024)
- [Learning Binary Residual Representations for Domain-specific Video Streaming](https://mlsanity.com/paper/1712.05087/) (2017)
- [OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression](https://mlsanity.com/paper/2107.09179/) (2021)
- [Pseudocylindrical Convolutions for Learned Omnidirectional Image Compression](https://mlsanity.com/paper/2112.13227/) (2021)
- [A Novel Diffusion Model for Pairwise Geoscience Data Generation with Unbalanced Training Dataset](https://mlsanity.com/paper/2501.00941/) (2025)
- [Motion-Plane-Adaptive Inter Prediction in 360-Degree Video Coding](https://mlsanity.com/paper/2202.03323/) (2022)
- [Motion Plane Adaptive Motion Modeling for Spherical Video Coding in H.266/VVC](https://mlsanity.com/paper/2306.13694/) (2023)

## Built on by (most cited first, 1 of 1)

- [Rate-Distortion Adaptive Primitive Selection for Omnidirectional Gaussian Splatting](https://mlsanity.com/paper/2609.34367/) (2026)

## Abstract (arXiv)

Given the popularity of 360deg images on social media platforms, 360deg image compression becomes a critical technology for media storage and transmission. Conventional 360deg image compression pipeline projects the spherical image into a single 2D plane, leading to issues of oversampling and distortion. In this paper, we propose a novel viewport-based neural compression pipeline for 360deg images. By replacing the image projection in conventional 360deg image compression pipelines with viewport extraction and efficiently compressing multiple viewports, the proposed pipeline minimizes the inherent oversampling and distortion issues. However, viewport extraction impedes information sharing between multiple viewports during compression, causing the loss of global information about the spherical image. To tackle this global information loss, we design a neural viewport codec to capture global prior information across multiple viewports and maximally compress the viewport data. The viewport codec is empowered by a transformer-based ViewPort ConText (VPCT) module that can be integrated with canonical learning-based 2D image compression structures. We compare the proposed pipeline with existing 360deg image compression models and conventional 360deg image compression pipelines building on learning-based 2D image codecs and standard hand-crafted codecs. Results show that our pipeline saves an average of 14.01% bit consumption compared to the best-performing 360deg image compression methods without compromising quality. The proposed VPCT-based codec also outperforms existing 2D image codecs in the viewport-based neural compression pipeline. Our code can be found at: https://github.com/Jingwei-Liao/VPCT.