mlsanityData to 14 Oct 2026

Structural and Statistical Texture Knowledge Distillation for Semantic Segmentation

arXiv 2305.03944 ↗First posted 6 May 2023cs.CVcited by 36

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

Existing knowledge distillation works for semantic segmentation mainly focus on transferring high-level contextual knowledge from teacher to student. However, low-level texture knowledge is also of vital importance for characterizing the local structural pattern and global statistical property, such as boundary, smoothness, regularity and color contrast, which may not be well addressed by high-level deep features. In this paper, we are intended to take full advantage of both structural and statistical texture knowledge and propose a novel Structural and Statistical Texture Knowledge Distillation (SSTKD) framework for semantic segmentation. Specifically, for structural texture knowledge, we introduce a Contourlet Decomposition Module (CDM) that decomposes low-level features with iterative Laplacian pyramid and directional filter bank to mine the structural texture knowledge. For statistical knowledge, we propose a Denoised Texture Intensity Equalization Module (DTIEM) to adaptively extract and enhance statistical texture knowledge through heuristics iterative quantization and denoised operation. Finally, each knowledge learning is supervised by an individual loss function, forcing the student network to mimic the teacher better from a broader perspective. Experiments show that the proposed method achieves state-of-the-art performance on Cityscapes, Pascal VOC 2012 and ADE20K datasets.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 33 of its 61 references that are in the map. First posted on arXiv 6 May 2023.

Builds on

33 of 61 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2017Graph Attention Networkscited by 8,923
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  9. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  10. 2016Pyramid Scene Parsing Networkcited by 2,479
  11. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
  12. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
21 more
  1. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  2. 2018Dual Attention Network for Scene Segmentationcited by 1,003
  3. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  4. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 576
  5. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  6. 2016RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentationcited by 465
  7. 2019PointRend: Image Segmentation as Renderingcited by 440
  8. 2016ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentationcited by 291
  9. 2019Structured Knowledge Distillation for Dense Predictioncited by 279
  10. 2017ICNet for Real-Time Semantic Segmentation on High-Resolution Imagescited by 261
  11. 2015Attention to Scale: Scale-aware Semantic Image Segmentationcited by 169
  12. 2020Channel-wise Knowledge Distillation for Dense Predictioncited by 163
  13. 2018OCNet: Object Context Network for Scene Parsingcited by 140
  14. 2019Knowledge Adaptation for Efficient Semantic Segmentationcited by 105
  15. 2021Learning Statistical Texture for Semantic Segmentationcited by 51
  16. 2020Class-wise Dynamic Graph Convolution for Semantic Segmentationcited by 44
  17. 2018Improving Fast Segmentation With Teacher-student Learningcited by 34
  18. 2014Feedforward semantic segmentation with zoom-out featurescited by 21
  19. 2020Context-Aware Graph Convolution Network for Target Re-identificationcited by 21
  20. 2016Using Filter Banks in Convolutional Neural Networks for Texture Classificationcited by 15
  21. 2016Exploring Context with Deep Structured models for Semantic Segmentationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 13; 2025: 8; 2026: 5.

36 papers.

Sort:

By area:

  1. 2024SAM2-Adapter: Evaluating & Adapting Segment Anything 2 in Downstream Tasks: Camouflage, Shadow, Medical Image Segmentation, and Morecited by 60
  2. 2023LLaFS: When Large Language Models Meet Few-Shot Segmentationcited by 48
  3. 2023A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentationcited by 33
  4. 2023Ultra-High Resolution Segmentation with Ultra-Rich Context: A Novel Benchmarkcited by 22
  5. 2025POPEN: Preference-Based Optimization and Ensemble for LVLM-Based Reasoning Segmentationcited by 22
  6. 2024Tree-of-Table: Unleashing the Power of LLMs for Enhanced Large-Scale Table Understandingcited by 20
  7. 2023Guided Patch-Grouping Wavelet Transformer with Spatial Congruence for Ultra-High Resolution Segmentationcited by 16
  8. 2024Discrete Latent Perspective Learning for Segmentation and Detectioncited by 16
  9. 2023BPKD: Boundary Privileged Knowledge Distillation For Semantic Segmentationcited by 15
  10. 2025ROS-SAM: High-Quality Interactive Segmentation for Remote Sensing Moving Objectcited by 15
  11. 2024View-Centric Multi-Object Tracking with Homographic Matching in Moving UAVcited by 10
  12. 2024PPTFormer: Pseudo Multi-Perspective Transformer for UAV Segmentationcited by 10
24 more
  1. 2025Structural and Statistical Texture Knowledge Distillation and Learning for Segmentationcited by 9
  2. 2023ChangeNet: Multi-Temporal Asymmetric Change Detection Datasetcited by 8
  3. 2025FreePCA: Integrating Consistency Information across Long-short Frames in Training-free Long Video Generation via Principal Component Analysiscited by 8
  4. 2025SAM3-Adapter: Efficient Adaptation of Segment Anything 3 for Camouflage Object Segmentation, Shadow Detection, and Medical Image Segmentationcited by 7
  5. 2023Histogram Layer Time Delay Neural Networks for Passive Sonar Classificationcited by 5
  6. 2023FerKD: Surgical Label Adaptation for Efficient Distillationcited by 5
  7. 2026StreamCacheVGGT: Streaming Visual Geometry Transformers with Robust Scoring and Hybrid Cache Compressioncited by 5
  8. 2024Histogram Layers for Neural Engineered Featurescited by 4
  9. 2024SSA-Seg: Semantic and Spatial Adaptive Pixel-level Classifier for Semantic Segmentationcited by 4
  10. 2025RAVEN++: Pinpointing Fine-Grained Violations in Advertisement Videos with Active Reinforcement Reasoningcited by 4
  11. 2026HD-VGGT: High-Resolution Visual Geometry Transformercited by 3
  12. 2026DeCo-DETR: Decoupled Cognition DETR for efficient Open-Vocabulary Object Detectioncited by 3
  13. 2024Weakly Supervised Change Detection via Knowledge Distillation and Multiscale Sigmoid Inferencecited by 2
  14. 2024Relative Difficulty Distillation for Semantic Segmentationcited by 2
  15. 2026Robust 4D Visual Geometry Transformer with Uncertainty-Aware Priorscited by 2
  16. 2023Towards Comparable Knowledge Distillation in Semantic Image Segmentationcited by 1
  17. 2025Distilled-3DGS:Distilled 3D Gaussian Splattingcited by 1
  18. 2023Bidirectional Knowledge Reconfiguration for Lightweight Point Cloud Analysiscited by 0
  19. 2024Generative Denoise Distillation: Simple Stochastic Noises Induce Efficient Knowledge Transfer for Dense Predictioncited by 0
  20. 2024Towards Customized Knowledge Distillation for Chip-Level Dense Image Predictionscited by 0
  21. 2024Investigation of Time-Frequency Feature Combinations with Histogram Layer Time Delay Neural Networkscited by 0
  22. 2024Hierarchical mutual distillation for multi-view fusion: Learning from all possible view combinationscited by 0
  23. 2025Structural and Statistical Audio Texture Knowledge Distillation for Acoustic Classificationcited by 0
  24. 2026ARGUS: Policy-Adaptive Ad Governance via Evolving Reinforcement with Adversarial Umpiringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024I2CKD : Intra- and Inter-Class Knowledge Distillation for Semantic Segmentation
  2. 2025Seeing the Whole Picture: Distribution-Guided Data-Free Distillation for Semantic Segmentation
  3. 2026The Surprising Effectiveness of Canonical Knowledge Distillation for Semantic Segmentation
  4. 2025Diffusion-Guided Knowledge Distillation for Weakly-Supervised Low-Light Semantic Segmentation
  5. 2021Double Similarity Distillation for Semantic Image Segmentation
  6. 2022Normalized Feature Distillation for Semantic Segmentation
  7. 2025Distilling Knowledge from Heterogeneous Architectures for Semantic Segmentation
  8. 2022Cross-Image Relational Knowledge Distillation for Semantic Segmentation