# Disentangled Latent Transformer for Interpretable Monocular Height Estimation

arXiv: https://arxiv.org/abs/2201.06357 · HTML: https://mlsanity.com/paper/2201.06357/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 17 Jan 2022.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [Enforcing geometric constraints of virtual normal for depth prediction](https://mlsanity.com/paper/1907.12209/) (2019)
- [Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals](https://mlsanity.com/paper/2102.06191/) (2021)
- [PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering](https://mlsanity.com/paper/2103.17070/) (2021)
- [Interpretable Convolutional Neural Networks](https://mlsanity.com/paper/1710.00935/) (2017)
- [Interpreting Super-Resolution Networks with Local Attribution Maps](https://mlsanity.com/paper/2011.11036/) (2020)
- [Deep learning in remote sensing: a review](https://mlsanity.com/paper/1710.03959/) (2017)
- [Semi-Supervised Deep Learning for Monocular Depth Map Prediction](https://mlsanity.com/paper/1702.02706/) (2017)
- [On the importance of single directions for generalization](https://mlsanity.com/paper/1803.06959/) (2018)
- [How do neural networks see depth in single images?](https://mlsanity.com/paper/1905.07005/) (2019)
- [The Edge of Depth: Explicit Constraints between Segmentation and Depth](https://mlsanity.com/paper/2004.00171/) (2020)
- [Semantic Stereo for Incidental Satellite Images](https://mlsanity.com/paper/1811.08739/) (2018)
- [IM2HEIGHT: Height Estimation from Single Monocular Imagery via Fully Residual Convolutional-Deconvolutional Network](https://mlsanity.com/paper/1802.10249/) (2018)
- [Training Interpretable Convolutional Neural Networks by Differentiating Class-specific Filters](https://mlsanity.com/paper/2007.08194/) (2020)
- [Visualization of Convolutional Neural Networks for Monocular Depth Estimation](https://mlsanity.com/paper/1904.03380/) (2019)
- [Learning Geocentric Object Pose in Oblique Monocular Images](https://mlsanity.com/paper/2007.00729/) (2020)
- [Towards Interpretable Deep Networks for Monocular Depth Estimation](https://mlsanity.com/paper/2108.05312/) (2021)
- [Single View Geocentric Pose in the Wild](https://mlsanity.com/paper/2105.08229/) (2021)
- [THE Benchmark: Transferable Representation Learning for Monocular Height Estimation](https://mlsanity.com/paper/2112.14985/) (2021)

## Built on by (most cited first, 3 of 3)

- [HTC-DC Net: Monocular Height Estimation from Single Remote Sensing Images](https://mlsanity.com/paper/2309.16486/) (2023)
- [Multilingual Augmentation for Robust Visual Question Answering in Remote Sensing Images](https://mlsanity.com/paper/2304.03844/) (2023)
- [Reconstructing Building Height from Spaceborne TomoSAR Point Clouds Using a Dual-Topology Network](https://mlsanity.com/paper/2601.00658/) (2026)

## Abstract (arXiv)

Monocular height estimation (MHE) from remote sensing imagery has high potential in generating 3D city models efficiently for a quick response to natural disasters. Most existing works pursue higher performance. However, there is little research exploring the interpretability of MHE networks. In this paper, we target at exploring how deep neural networks predict height from a single monocular image. Towards a comprehensive understanding of MHE networks, we propose to interpret them from multiple levels: 1) Neurons: unit-level dissection. Exploring the semantic and height selectivity of the learned internal deep representations; 2) Instances: object-level interpretation. Studying the effects of different semantic classes, scales, and spatial contexts on height estimation; 3) Attribution: pixel-level analysis. Understanding which input pixels are important for the height estimation. Based on the multi-level interpretation, a disentangled latent Transformer network is proposed towards a more compact, reliable, and explainable deep model for monocular height estimation. Furthermore, a novel unsupervised semantic segmentation task based on height estimation is first introduced in this work. Additionally, we also construct a new dataset for joint semantic segmentation and height estimation. Our work provides novel insights for both understanding and designing MHE models.