# Distilled Pooling Transformer Encoder for Efficient Realistic Image Dehazing

arXiv: https://arxiv.org/abs/2412.14220 · HTML: https://mlsanity.com/paper/2412.14220/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 58 references that are in the map.
- First posted on arXiv 18 Dec 2024.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer](https://mlsanity.com/paper/2110.02178/) (2021)
- [MetaFormer Is Actually What You Need for Vision](https://mlsanity.com/paper/2111.11418/) (2021)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [DehazeNet: An End-to-End System for Single Image Haze Removal](https://mlsanity.com/paper/1601.07661/) (2016)
- [FFA-Net: Feature Fusion Attention Network for Single Image Dehazing](https://mlsanity.com/paper/1911.07559/) (2019)
- [GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing](https://mlsanity.com/paper/1908.03245/) (2019)
- [Contrastive Learning for Compact Single Image Dehazing](https://mlsanity.com/paper/2104.09367/) (2021)
- [Gated Fusion Network for Single Image Dehazing](https://mlsanity.com/paper/1804.00213/) (2018)
- [EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers](https://mlsanity.com/paper/2205.03436/) (2022)
- [NH-HAZE: An Image Dehazing Benchmark with Non-Homogeneous Hazy and Haze-Free Images](https://mlsanity.com/paper/2005.03560/) (2020)
- [Dense Haze: A benchmark for image dehazing with dense-haze and haze-free images](https://mlsanity.com/paper/1904.02904/) (2019)
- [O-HAZE: a dehazing benchmark with real hazy and haze-free outdoor images](https://mlsanity.com/paper/1804.05101/) (2018)
- [Gated Context Aggregation Network for Image Dehazing and Deraining](https://mlsanity.com/paper/1811.08747/) (2018)
- [Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing](https://mlsanity.com/paper/1805.05308/) (2018)
- [I-HAZE: a dehazing benchmark with real hazy and haze-free indoor images](https://mlsanity.com/paper/1804.05091/) (2018)
- [Deploying Image Deblurring across Mobile Devices: A Perspective of Quality and Latency](https://mlsanity.com/paper/2004.12599/) (2020)
- [A Novel Encoder-Decoder Network with Guided Transmission Map for Single Image Dehazing](https://mlsanity.com/paper/2202.04757/) (2022)
- [Single Image Dehazing with An Independent Detail-Recovery Network](https://mlsanity.com/paper/2109.10492/) (2021)
- [Rich Feature Distillation with Feature Affinity Module for Efficient Image Dehazing](https://mlsanity.com/paper/2207.11250/) (2022)

## Built on by (most cited first, 3 of 3)

- [Low-Light Enhancement via Encoder-Decoder Network with Illumination Guidance](https://mlsanity.com/paper/2507.13360/) (2025)
- [MetaFormer-driven Encoding Network for Robust Medical Semantic Segmentation](https://mlsanity.com/paper/2601.00922/) (2026)
- [Exploring SAM Supervision for Fine-Grained UAV Target Segmentation under Data Scarcity](https://mlsanity.com/paper/2607.03754/) (2026)

## Abstract (arXiv)

This paper proposes a lightweight neural network designed for realistic image dehazing, utilizing a Distilled Pooling Transformer Encoder, named DPTE-Net. Recently, while vision transformers (ViTs) have achieved great success in various vision tasks, their self-attention (SA) module's complexity scales quadratically with image resolution, hindering their applicability on resource-constrained devices. To overcome this, the proposed DPTE-Net substitutes traditional SA modules with efficient pooling mechanisms, significantly reducing computational demands while preserving ViTs' learning capabilities. To further enhance semantic feature learning, a distillation-based training process is implemented which transfers rich knowledge from a larger teacher network to DPTE-Net. Additionally, DPTE-Net is trained within a generative adversarial network (GAN) framework, leveraging the strong generalization of GAN in image restoration, and employs a transmission-aware loss function to dynamically adapt to varying haze densities. Experimental results on various benchmark datasets have shown that the proposed DPTE-Net can achieve competitive dehazing performance when compared to state-of-the-art methods while maintaining low computational complexity, making it a promising solution for resource-limited applications. The code of this work is available at https://github.com/tranleanh/dpte-net.