# SelfReformer: Self-Refined Network with Transformer for Salient Object Detection

arXiv: https://arxiv.org/abs/2205.11283 · HTML: https://mlsanity.com/paper/2205.11283/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 51 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [EGNet:Edge Guidance Network for Salient Object Detection](https://mlsanity.com/paper/1908.08297/) (2019)
- [A Simple Pooling-Based Design for Real-Time Salient Object Detection](https://mlsanity.com/paper/1904.09569/) (2019)
- [Multi-scale Interactive Network for Salient Object Detection](https://mlsanity.com/paper/2007.09062/) (2020)
- [The Secrets of Salient Object Segmentation](https://mlsanity.com/paper/1406.2807/) (2014)
- [Deeply supervised salient object detection with short connections](https://mlsanity.com/paper/1611.04849/) (2016)
- [Visual Saliency Based on Multiscale Deep Features](https://mlsanity.com/paper/1503.08663/) (2015)
- [HOTR: End-to-End Human-Object Interaction Detection with Transformers](https://mlsanity.com/paper/2104.13682/) (2021)
- [F3Net: Fusion, Feedback and Focus for Salient Object Detection](https://mlsanity.com/paper/1911.11445/) (2019)
- [Visual Saliency Transformer](https://mlsanity.com/paper/2104.12099/) (2021)
- [Suppress and Balance: A Simple Gated Network for Salient Object Detection](https://mlsanity.com/paper/2007.08074/) (2020)
- [Diverse Part Discovery: Occluded Person Re-identification with Part-Aware Transformer](https://mlsanity.com/paper/2106.04095/) (2021)
- [Label Decoupling Framework for Salient Object Detection](https://mlsanity.com/paper/2008.11048/) (2020)
- [From Captions to Visual Concepts and Back](https://mlsanity.com/paper/1411.4952/) (2014)
- [Global Context-Aware Progressive Aggregation Network for Salient Object Detection](https://mlsanity.com/paper/2003.00651/) (2020)

## Built on by (most cited first, 20 of 20)

- [VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks](https://mlsanity.com/paper/2406.08394/) (2024)
- [X-Paste: Revisiting Scalable Copy-Paste for Instance Segmentation using CLIP and StableDiffusion](https://mlsanity.com/paper/2212.03863/) (2022)
- [Object-Centric Diffusion for Efficient Video Editing](https://mlsanity.com/paper/2401.05735/) (2024)
- [A Simple yet Effective Network based on Vision Transformer for Camouflaged Object and Salient Object Detection](https://mlsanity.com/paper/2402.18922/) (2024)
- [Explicit Visual Prompting for Universal Foreground Segmentations](https://mlsanity.com/paper/2305.18476/) (2023)
- [DiverGen: Improving Instance Segmentation by Learning Wider Data Distribution with More Diverse Generative Data](https://mlsanity.com/paper/2405.10185/) (2024)
- [DC-Net: Divide-and-Conquer for Salient Object Detection](https://mlsanity.com/paper/2305.14955/) (2023)
- [Gen2Det: Generate to Detect](https://mlsanity.com/paper/2312.04566/) (2023)
- [Generative Active Learning for Long-tailed Instance Segmentation](https://mlsanity.com/paper/2406.02435/) (2024)
- [Simultaneous Image-to-Zero and Zero-to-Noise: Diffusion Models with Analytical Image Attenuation](https://mlsanity.com/paper/2306.13720/) (2023)
- [M3Net: Multilevel, Mixed and Multistage Attention Network for Salient Object Detection](https://mlsanity.com/paper/2309.08365/) (2023)
- [SEMPART: Self-supervised Multi-resolution Partitioning of Image Semantics](https://mlsanity.com/paper/2309.10972/) (2023)
- [Distribution-Specific Learning for Joint Salient and Camouflaged Object Detection](https://mlsanity.com/paper/2508.06063/) (2025)
- [Building Flyweight FLIM-based CNNs with Adaptive Decoding for Object Detection](https://mlsanity.com/paper/2306.14840/) (2023)
- [External Prompt Features Enhanced Parameter-efficient Fine-tuning for Salient Object Detection](https://mlsanity.com/paper/2404.15008/) (2024)
- [A positive feedback method based on F-measure value for Salient Object Detection](https://mlsanity.com/paper/2304.14619/) (2023)
- [Saliency-R1: Incentivizing Unified Saliency Reasoning Capability in MLLM with Confidence-Guided Reinforcement Learning](https://mlsanity.com/paper/2511.00396/) (2025)
- [Revisiting Salient Object Detection from an Observer-Centric Perspective](https://mlsanity.com/paper/2602.06369/) (2026)
- [IP-SAM: Rethinking Prompt-Conditioned Segmentation for Prompt-Absent Deployment](https://mlsanity.com/paper/2603.27250/) (2026)
- [Global-Local Feature Decoding with Adapter-Guided SAMv2 for Salient Object Detection](https://mlsanity.com/paper/2605.02616/) (2026)

## Abstract (arXiv)

The global and local contexts significantly contribute to the integrity of predictions in Salient Object Detection (SOD). Unfortunately, existing methods still struggle to generate complete predictions with fine details. There are two major problems in conventional approaches: first, for global context, high-level CNN-based encoder features cannot effectively catch long-range dependencies, resulting in incomplete predictions. Second, downsampling the ground truth to fit the size of predictions will introduce inaccuracy as the ground truth details are lost during interpolation or pooling. Thus, in this work, we developed a Transformer-based network and framed a supervised task for a branch to learn the global context information explicitly. Besides, we adopt Pixel Shuffle from Super-Resolution (SR) to reshape the predictions back to the size of ground truth instead of the reverse. Thus details in the ground truth are untouched. In addition, we developed a two-stage Context Refinement Module (CRM) to fuse global context and automatically locate and refine the local details in the predictions. The proposed network can guide and correct itself based on the global and local context generated, thus is named, Self-Refined Transformer (SelfReformer). Extensive experiments and evaluation results on five benchmark datasets demonstrate the outstanding performance of the network, and we achieved the state-of-the-art.