# DepthTCM: High Efficient Depth Compression via Physics-aware Transformer-CNN Mixed Architecture

arXiv: https://arxiv.org/abs/2603.21233 · HTML: https://mlsanity.com/paper/2603.21233/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 22 Mar 2026.

## Builds on (most cited first, 15 of 15)

- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [DIODE: A Dense Indoor and Outdoor DEpth Dataset](https://mlsanity.com/paper/1908.00463/) (2019)
- [Learned Image Compression with Mixed Transformer-CNN Architectures](https://mlsanity.com/paper/2303.14978/) (2023)
- [The Devil Is in the Details: Window-based Attention for Image Compression](https://mlsanity.com/paper/2203.08450/) (2022)
- [Entroformer: A Transformer-based Entropy Model for Learned Image Compression](https://mlsanity.com/paper/2202.05492/) (2022)
- [COOL-CHIC: Coordinate-based Low Complexity Hierarchical Image Codec](https://mlsanity.com/paper/2212.05458/) (2022)
- [Contextformer: A Transformer with Spatio-Channel Attention for Context Modeling in Learned Image Compression](https://mlsanity.com/paper/2203.02452/) (2022)
- [End-to-end lossless compression of high precision depth maps guided by pseudo-residual](https://mlsanity.com/paper/2201.03195/) (2022)
- [Neural Network Assisted Depth Map Packing for Compression Using Standard Hardware Video Codecs](https://mlsanity.com/paper/2206.15183/) (2022)
- [Range Image-Based Implicit Neural Compression for LiDAR Point Clouds](https://mlsanity.com/paper/2504.17229/) (2025)

## Abstract (arXiv)

We propose DepthTCM, a physics-aware end-to-end framework for depth map compression. In our framework of DepthTCM, the high-bit depth map is first converted to a conventional 3-channel image representation losslessly using a method inspired by a physical sinusoidal fringe pattern based profiliometry system, then the 3-channel color image is encoded and decoded by a recently developed Transformer-CNN mixed neural network architecture. Specifically, DepthTCM maps depth to a smooth 3-channel using multiwavelength depth (MWD) encoding, then globally quantized the MWD encoded representation to 4 bits per channel to reduce entropy, and finally is compressed using a learned codec that combines convolutional and Transformer layers. Experiment results demonstrate the advantage of our proposed method. On Middlebury 2014, DepthTCM reaches 0.307 bpp while preserving 99.38% accuracy, a level of fidelity commensurate with lossless PNG. We additionally demonstrate practical efficiency and scalability, reporting average end-to-end inference times of 41.48 ms (encoder) and 47.45 ms (decoder) on the ScanNet++ iPhone RGB-D subset. Ablations validate our design choices: relative to 8-bit quantization, 4-bit quantization reduces bitrate by 66% while maintaining comparable reconstruction quality, with only a marginal 0.68 dB PSNR change and a 0.04% accuracy difference. In addition, Transformer--CNN blocks further improve PSNR by up to 0.75 dB over CNN-only architectures.