# Rethinking Object Saliency Ranking: A Novel Whole-flow Processing Paradigm

arXiv: https://arxiv.org/abs/2312.03226 · HTML: https://mlsanity.com/paper/2312.03226/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 58 references that are in the map.
- First posted on arXiv 6 Dec 2023.

## Builds on (most cited first, 28 of 28)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020)
- [Concealed Object Detection](https://mlsanity.com/paper/2102.10274/) (2021)
- [Simultaneously Localize, Segment and Rank the Camouflaged Objects](https://mlsanity.com/paper/2103.04011/) (2021)
- [Salient Object Detection in the Deep Learning Era: An In-Depth Survey](https://mlsanity.com/paper/1904.09146/) (2019)
- [Salient Object Detection via Integrity Learning](https://mlsanity.com/paper/2101.07663/) (2021)
- [Unified Image and Video Saliency Modeling](https://mlsanity.com/paper/2003.05477/) (2020)
- [Deep Visual Attention Prediction](https://mlsanity.com/paper/1705.02544/) (2017)
- [SalGAN: Visual Saliency Prediction with Generative Adversarial Networks](https://mlsanity.com/paper/1701.01081/) (2017)
- [Uncertainty Inspired RGB-D Saliency Detection](https://mlsanity.com/paper/2009.03075/) (2020)
- [A Deep Multi-Level Network for Saliency Prediction](https://mlsanity.com/paper/1609.01064/) (2016)
- [Re-thinking Co-Salient Object Detection](https://mlsanity.com/paper/2007.03380/) (2020)
- [EML-NET:An Expandable Multi-Layer NETwork for Saliency Prediction](https://mlsanity.com/paper/1805.01047/) (2018)
- [RGB-D Saliency Detection via Cascaded Mutual Information Minimization](https://mlsanity.com/paper/2109.07246/) (2021)
- [Advances in Deep Concealed Scene Understanding](https://mlsanity.com/paper/2304.11234/) (2023)
- [TranSalNet: Towards perceptually relevant visual saliency prediction](https://mlsanity.com/paper/2110.03593/) (2021)
- [Revisiting Salient Object Detection: Simultaneous Detection, Ranking, and Subitizing of Multiple Salient Objects](https://mlsanity.com/paper/1803.05082/) (2018)
- [Personal Fixations-Based Object Segmentation with Object Localization and Boundary Preservation](https://mlsanity.com/paper/2101.09014/) (2021)
- [Relative Saliency and Ranking: Models, Metrics, Data, and Benchmarks](https://mlsanity.com/paper/1810.02426/) (2018)
- [Salient Object Ranking with Position-Preserved Attention](https://mlsanity.com/paper/2106.05047/) (2021)
- [Instance-Level Relative Saliency Ranking with Graph Reasoning](https://mlsanity.com/paper/2107.03824/) (2021)
- [SalFBNet: Learning Pseudo-Saliency Distribution via Feedback Convolutional Networks](https://mlsanity.com/paper/2112.03731/) (2021)
- [Bi-directional Object-context Prioritization Learning for Saliency Ranking](https://mlsanity.com/paper/2203.09416/) (2022)
- [Modeling Long-Term Memory and Temporal Attention Shifts for Video Salient Object Ranking with a New Benchmark](https://mlsanity.com/paper/2203.17257/) (2022)

## Built on by (most cited first, 4 of 4)

- [Quality-aware Selective Fusion Network for V-D-T Salient Object Detection](https://mlsanity.com/paper/2405.07655/) (2024)
- [Unveiling Context-Related Anomalies: Knowledge Graph Empowered Decoupling of Scene and Action for Human-Related Video Anomaly Detection](https://mlsanity.com/paper/2409.03236/) (2024)
- [HiRQA: Hierarchical Ranking and Quality Alignment for Opinion-Unaware Image Quality Assessment](https://mlsanity.com/paper/2508.15130/) (2025)
- [Camera-based implicit mind reading by capturing higher-order semantic dynamics of human gaze within environmental context](https://mlsanity.com/paper/2507.12889/) (2025)

## Abstract (arXiv)

Existing salient object detection methods are capable of predicting binary maps that highlight visually salient regions. However, these methods are limited in their ability to differentiate the relative importance of multiple objects and the relationships among them, which can lead to errors and reduced accuracy in downstream tasks that depend on the relative importance of multiple objects. To conquer, this paper proposes a new paradigm for saliency ranking, which aims to completely focus on ranking salient objects by their "importance order". While previous works have shown promising performance, they still face ill-posed problems. First, the saliency ranking ground truth (GT) orders generation methods are unreasonable since determining the correct ranking order is not well-defined, resulting in false alarms. Second, training a ranking model remains challenging because most saliency ranking methods follow the multi-task paradigm, leading to conflicts and trade-offs among different tasks. Third, existing regression-based saliency ranking methods are complex for saliency ranking models due to their reliance on instance mask-based saliency ranking orders. These methods require a significant amount of data to perform accurately and can be challenging to implement effectively. To solve these problems, this paper conducts an in-depth analysis of the causes and proposes a whole-flow processing paradigm of saliency ranking task from the perspective of "GT data generation", "network structure design" and "training protocol". The proposed approach outperforms existing state-of-the-art methods on the widely-used SALICON set, as demonstrated by extensive experiments with fair and reasonable comparisons. The saliency ranking task is still in its infancy, and our proposed unified framework can serve as a fundamental strategy to guide future work.