# Structural and Statistical Texture Knowledge Distillation for Semantic Segmentation

arXiv: https://arxiv.org/abs/2305.03944 · HTML: https://mlsanity.com/paper/2305.03944/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 33 of its 61 references that are in the map.
- First posted on arXiv 6 May 2023.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/1611.06612/) (2016)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation](https://mlsanity.com/paper/1606.02147/) (2016)
- [Structured Knowledge Distillation for Dense Prediction](https://mlsanity.com/paper/1903.04197/) (2019)
- [ICNet for Real-Time Semantic Segmentation on High-Resolution Images](https://mlsanity.com/paper/1704.08545/) (2017)
- [Attention to Scale: Scale-aware Semantic Image Segmentation](https://mlsanity.com/paper/1511.03339/) (2015)
- [Channel-wise Knowledge Distillation for Dense Prediction](https://mlsanity.com/paper/2011.13256/) (2020)
- [OCNet: Object Context Network for Scene Parsing](https://mlsanity.com/paper/1809.00916/) (2018)
- [Knowledge Adaptation for Efficient Semantic Segmentation](https://mlsanity.com/paper/1903.04688/) (2019)
- [Learning Statistical Texture for Semantic Segmentation](https://mlsanity.com/paper/2103.04133/) (2021)
- [Class-wise Dynamic Graph Convolution for Semantic Segmentation](https://mlsanity.com/paper/2007.09690/) (2020)
- [Improving Fast Segmentation With Teacher-student Learning](https://mlsanity.com/paper/1810.08476/) (2018)
- [Feedforward semantic segmentation with zoom-out features](https://mlsanity.com/paper/1412.0774/) (2014)

## Built on by (most cited first, 30 of 36)

- [SAM2-Adapter: Evaluating & Adapting Segment Anything 2 in Downstream Tasks: Camouflage, Shadow, Medical Image Segmentation, and More](https://mlsanity.com/paper/2408.04579/) (2024)
- [LLaFS: When Large Language Models Meet Few-Shot Segmentation](https://mlsanity.com/paper/2311.16926/) (2023)
- [A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentation](https://mlsanity.com/paper/2307.12574/) (2023)
- [Ultra-High Resolution Segmentation with Ultra-Rich Context: A Novel Benchmark](https://mlsanity.com/paper/2305.10899/) (2023)
- [POPEN: Preference-Based Optimization and Ensemble for LVLM-Based Reasoning Segmentation](https://mlsanity.com/paper/2504.00640/) (2025)
- [Tree-of-Table: Unleashing the Power of LLMs for Enhanced Large-Scale Table Understanding](https://mlsanity.com/paper/2411.08516/) (2024)
- [Guided Patch-Grouping Wavelet Transformer with Spatial Congruence for Ultra-High Resolution Segmentation](https://mlsanity.com/paper/2307.00711/) (2023)
- [Discrete Latent Perspective Learning for Segmentation and Detection](https://mlsanity.com/paper/2406.10475/) (2024)
- [BPKD: Boundary Privileged Knowledge Distillation For Semantic Segmentation](https://mlsanity.com/paper/2306.08075/) (2023)
- [ROS-SAM: High-Quality Interactive Segmentation for Remote Sensing Moving Object](https://mlsanity.com/paper/2503.12006/) (2025)
- [View-Centric Multi-Object Tracking with Homographic Matching in Moving UAV](https://mlsanity.com/paper/2403.10830/) (2024)
- [PPTFormer: Pseudo Multi-Perspective Transformer for UAV Segmentation](https://mlsanity.com/paper/2406.19632/) (2024)
- [Structural and Statistical Texture Knowledge Distillation and Learning for Segmentation](https://mlsanity.com/paper/2503.08043/) (2025)
- [ChangeNet: Multi-Temporal Asymmetric Change Detection Dataset](https://mlsanity.com/paper/2312.17428/) (2023)
- [FreePCA: Integrating Consistency Information across Long-short Frames in Training-free Long Video Generation via Principal Component Analysis](https://mlsanity.com/paper/2505.01172/) (2025)
- [SAM3-Adapter: Efficient Adaptation of Segment Anything 3 for Camouflage Object Segmentation, Shadow Detection, and Medical Image Segmentation](https://mlsanity.com/paper/2511.19425/) (2025)
- [Histogram Layer Time Delay Neural Networks for Passive Sonar Classification](https://mlsanity.com/paper/2307.13788/) (2023)
- [FerKD: Surgical Label Adaptation for Efficient Distillation](https://mlsanity.com/paper/2312.17473/) (2023)
- [StreamCacheVGGT: Streaming Visual Geometry Transformers with Robust Scoring and Hybrid Cache Compression](https://mlsanity.com/paper/2604.15237/) (2026)
- [Histogram Layers for Neural Engineered Features](https://mlsanity.com/paper/2403.17176/) (2024)
- [SSA-Seg: Semantic and Spatial Adaptive Pixel-level Classifier for Semantic Segmentation](https://mlsanity.com/paper/2405.06525/) (2024)
- [RAVEN++: Pinpointing Fine-Grained Violations in Advertisement Videos with Active Reinforcement Reasoning](https://mlsanity.com/paper/2511.19168/) (2025)
- [HD-VGGT: High-Resolution Visual Geometry Transformer](https://mlsanity.com/paper/2603.27222/) (2026)
- [DeCo-DETR: Decoupled Cognition DETR for efficient Open-Vocabulary Object Detection](https://mlsanity.com/paper/2604.02753/) (2026)
- [Weakly Supervised Change Detection via Knowledge Distillation and Multiscale Sigmoid Inference](https://mlsanity.com/paper/2403.05796/) (2024)
- [Relative Difficulty Distillation for Semantic Segmentation](https://mlsanity.com/paper/2407.03719/) (2024)
- [Robust 4D Visual Geometry Transformer with Uncertainty-Aware Priors](https://mlsanity.com/paper/2604.09366/) (2026)
- [Towards Comparable Knowledge Distillation in Semantic Image Segmentation](https://mlsanity.com/paper/2309.03659/) (2023)
- [Distilled-3DGS:Distilled 3D Gaussian Splatting](https://mlsanity.com/paper/2508.14037/) (2025)
- [Bidirectional Knowledge Reconfiguration for Lightweight Point Cloud Analysis](https://mlsanity.com/paper/2310.05125/) (2023)

## Abstract (arXiv)

Existing knowledge distillation works for semantic segmentation mainly focus on transferring high-level contextual knowledge from teacher to student. However, low-level texture knowledge is also of vital importance for characterizing the local structural pattern and global statistical property, such as boundary, smoothness, regularity and color contrast, which may not be well addressed by high-level deep features. In this paper, we are intended to take full advantage of both structural and statistical texture knowledge and propose a novel Structural and Statistical Texture Knowledge Distillation (SSTKD) framework for semantic segmentation. Specifically, for structural texture knowledge, we introduce a Contourlet Decomposition Module (CDM) that decomposes low-level features with iterative Laplacian pyramid and directional filter bank to mine the structural texture knowledge. For statistical knowledge, we propose a Denoised Texture Intensity Equalization Module (DTIEM) to adaptively extract and enhance statistical texture knowledge through heuristics iterative quantization and denoised operation. Finally, each knowledge learning is supervised by an individual loss function, forcing the student network to mimic the teacher better from a broader perspective. Experiments show that the proposed method achieves state-of-the-art performance on Cityscapes, Pascal VOC 2012 and ADE20K datasets.