# Hard Pixel Mining for Depth Privileged Semantic Segmentation

arXiv: https://arxiv.org/abs/1906.11437 · HTML: https://mlsanity.com/paper/1906.11437/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jun 2019.

## Built on by (most cited first, 13 of 13)

- [Visible Watermark Removal via Self-calibrated Localization and Background Refinement](https://mlsanity.com/paper/2108.03581/) (2021)
- [When Semantic Segmentation Meets Frequency Aliasing](https://mlsanity.com/paper/2403.09065/) (2024)
- [Pyramid Fusion Transformer for Semantic Segmentation](https://mlsanity.com/paper/2201.04019/) (2022)
- [Contour-Aware Equipotential Learning for Semantic Segmentation](https://mlsanity.com/paper/2210.00223/) (2022)
- [Inharmonious Region Localization](https://mlsanity.com/paper/2104.09453/) (2021)
- [MutualFormer: Multi-Modality Representation Learning via Cross-Diffusion Attention](https://mlsanity.com/paper/2112.01177/) (2021)
- [BFANet: Revisiting 3D Semantic Segmentation with Boundary Feature Analysis](https://mlsanity.com/paper/2503.12539/) (2025)
- [GEM: Boost Simple Network for Glass Surface Segmentation via Vision Foundation Models](https://mlsanity.com/paper/2307.12018/) (2023)
- [Accelerating the creation of instance segmentation training sets through bounding box annotation](https://mlsanity.com/paper/2205.11563/) (2022)
- [Hard Region Aware Network for Remote Sensing Change Detection](https://mlsanity.com/paper/2305.19513/) (2023)
- [Optimizing against Infeasible Inclusions from Data for Semantic Segmentation through Morphology](https://mlsanity.com/paper/2408.14672/) (2024)
- [GeomPrompt: Geometric Prompt Learning for RGB-D Semantic Segmentation Under Missing and Degraded Depth](https://mlsanity.com/paper/2604.11585/) (2026)
- [Rare-Aware Autoencoding: Reconstructing Spatially Imbalanced Data](https://mlsanity.com/paper/2604.02031/) (2026)

## Abstract (arXiv)

Semantic segmentation has achieved remarkable progress but remains challenging due to the complex scene, object occlusion, and so on. Some research works have attempted to use extra information such as a depth map to help RGB based semantic segmentation because the depth map could provide complementary geometric cues. However, due to the inaccessibility of depth sensors, depth information is usually unavailable for the test images. In this paper, we leverage only the depth of training images as the privileged information to mine the hard pixels in semantic segmentation, in which depth information is only available for training images but not available for test images. Specifically, we propose a novel Loss Weight Module, which outputs a loss weight map by employing two depth-related measurements of hard pixels: Depth Prediction Error and Depthaware Segmentation Error. The loss weight map is then applied to segmentation loss, with the goal of learning a more robust model by paying more attention to the hard pixels. Besides, we also explore a curriculum learning strategy based on the loss weight map. Meanwhile, to fully mine the hard pixels on different scales, we apply our loss weight module to multi-scale side outputs. Our hard pixels mining method achieves the state-of-the-art results on two benchmark datasets, and even outperforms the methods which need depth input during testing.