# Deep Learning-based Compressed Domain Multimedia for Man and Machine: A Taxonomy and Application to Point Cloud Classification

arXiv: https://arxiv.org/abs/2310.18849 · HTML: https://mlsanity.com/paper/2310.18849/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 56 references that are in the map.
- First posted on arXiv 28 Oct 2023.

## Builds on (most cited first, 19 of 19)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data](https://mlsanity.com/paper/1908.04616/) (2019)
- [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds](https://mlsanity.com/paper/2203.11139/) (2022)
- [Multiscale Point Cloud Geometry Compression](https://mlsanity.com/paper/2011.03799/) (2020)
- [Sparse Tensor-based Multiscale Representation for Point Cloud Geometry Compression](https://mlsanity.com/paper/2111.10633/) (2021)
- [Improved Deep Point Cloud Geometry Compression](https://mlsanity.com/paper/2006.09043/) (2020)
- [IT/IST/IPLeiria Response to the Call for Proposals on JPEG Pleno Point Cloud Coding](https://mlsanity.com/paper/2208.02716/) (2022)
- [Learning-based Compression for Material and Texture Recognition](https://mlsanity.com/paper/2104.10065/) (2021)

## Built on by (most cited first, 4 of 4)

- [The JPEG Pleno Learning-based Point Cloud Coding Standard: Serving Man and Machine](https://mlsanity.com/paper/2409.08130/) (2024)
- [Point Cloud Geometry Scalable Coding with a Quality-Conditioned Latents Probability Estimator](https://mlsanity.com/paper/2404.07698/) (2024)
- [A Double Deep Learning-based Solution for Efficient Event Data Coding and Classification](https://mlsanity.com/paper/2407.15531/) (2024)
- [Point Cloud Geometry Scalable Coding Using a Resolution and Quality-conditioned Latents Probability Estimator](https://mlsanity.com/paper/2502.14099/) (2025)

## Abstract (arXiv)

In the current golden age of multimedia, human visualization is no longer the single main target, with the final consumer often being a machine which performs some processing or computer vision tasks. In both cases, deep learning plays a undamental role in extracting features from the multimedia representation data, usually producing a compressed representation referred to as latent representation. The increasing development and adoption of deep learning-based solutions in a wide area of multimedia applications have opened an exciting new vision where a common compressed multimedia representation is used for both man and machine. The main benefits of this vision are two-fold: i) improved performance for the computer vision tasks, since the effects of coding artifacts are mitigated; and ii) reduced computational complexity, since prior decoding is not required. This paper proposes the first taxonomy for designing compressed domain computer vision solutions driven by the architecture and weights compatibility with an available spatio-temporal computer vision processor. The potential of the proposed taxonomy is demonstrated for the specific case of point cloud classification by designing novel compressed domain processors using the JPEG Pleno Point Cloud Coding standard under development and adaptations of the PointGrid classifier. Experimental results show that the designed compressed domain point cloud classification solutions can significantly outperform the spatial-temporal domain classification benchmarks when applied to the decompressed data, containing coding artifacts, and even surpass their performance when applied to the original uncompressed data.