# TinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation

arXiv: https://arxiv.org/abs/2301.04619 · HTML: https://mlsanity.com/paper/2301.04619/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 11 Jan 2023.

## Builds on (most cited first, 26 of 26)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Improved Knowledge Distillation via Teacher Assistant](https://mlsanity.com/paper/1902.03393/) (2019)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Data](https://mlsanity.com/paper/2008.06180/) (2020)
- [Unified Image and Video Saliency Modeling](https://mlsanity.com/paper/2003.05477/) (2020)
- [Deep Visual Attention Prediction](https://mlsanity.com/paper/1705.02544/) (2017)
- [DADA: Driver Attention Prediction in Driving Accident Scenarios](https://mlsanity.com/paper/1912.12148/) (2019)
- [FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning](https://mlsanity.com/paper/2102.02514/) (2021)
- [TASED-Net: Temporally-Aggregating Spatial Encoder-Decoder Network for Video Saliency Detection](https://mlsanity.com/paper/1908.05786/) (2019)
- [ViNet: Pushing the limits of Visual Modality for Audio-Visual Saliency Prediction](https://mlsanity.com/paper/2012.06170/) (2020)
- [Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition](https://mlsanity.com/paper/1312.7570/) (2013)
- [Simple vs complex temporal recurrences for video saliency prediction](https://mlsanity.com/paper/1907.01869/) (2019)
- [Hierarchical Domain-Adapted Feature Learning for Video Saliency Prediction](https://mlsanity.com/paper/2010.01220/) (2020)
- [Temporal-Spatial Feature Pyramid for Video Saliency Detection](https://mlsanity.com/paper/2105.04213/) (2021)
- [GAZED- Gaze-guided Cinematic Editing of Wide-Angle Monocular Video Recordings](https://mlsanity.com/paper/2010.11886/) (2020)
- [FastSal: a Computationally Efficient Network for Visual Saliency Prediction](https://mlsanity.com/paper/2008.11151/) (2020)
- [Ultrafast Video Attention Prediction with Coupled Knowledge Distillation](https://mlsanity.com/paper/1904.04449/) (2019)
- [Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency](https://mlsanity.com/paper/1904.04992/) (2019)
- [RecSal : Deep Recursive Supervision for Visual Saliency Prediction](https://mlsanity.com/paper/2008.13745/) (2020)
- [Fast Video Salient Object Detection via Spatiotemporal Knowledge Distillation](https://mlsanity.com/paper/2010.10027/) (2020)

## Built on by (most cited first, 9 of 9)

- [DiffSal: Joint Audio and Video Learning for Diffusion Saliency Prediction](https://mlsanity.com/paper/2403.01226/) (2024)
- [Reframe Anything: LLM Agent for Open World Video Reframing](https://mlsanity.com/paper/2403.06070/) (2024)
- [Minimalistic Video Saliency Prediction via Efficient Decoder & Spatio Temporal Action Cues](https://mlsanity.com/paper/2502.00397/) (2025)
- [UniST: Towards Unifying Saliency Transformer for Video Saliency Prediction and Detection](https://mlsanity.com/paper/2309.08220/) (2023)
- [Facilitating the Production of Well-tailored Video Summaries for Sharing on Social Media](https://mlsanity.com/paper/2312.02616/) (2023)
- [Selective Attention-based Modulation for Continual Learning](https://mlsanity.com/paper/2403.20086/) (2024)
- [DTFSal: Audio-Visual Dynamic Token Fusion for Video Saliency Prediction](https://mlsanity.com/paper/2504.10070/) (2025)
- [SED:Lightweight Saliency prediction for Event-based data via Distillation](https://mlsanity.com/paper/2606.14631/) (2026)
- [Moving Forward with Video Saliency: A New Dataset and Benchmark where Motion Matters](https://mlsanity.com/paper/2610.03276/) (2026)

## Abstract (arXiv)

Video saliency prediction has recently attracted attention of the research community, as it is an upstream task for several practical applications. However, current solutions are particularly computationally demanding, especially due to the wide usage of spatio-temporal 3D convolutions. We observe that, while different model architectures achieve similar performance on benchmarks, visual variations between predicted saliency maps are still significant. Inspired by this intuition, we propose a lightweight model that employs multiple simple heterogeneous decoders and adopts several practical approaches to improve accuracy while keeping computational costs low, such as hierarchical multi-map knowledge distillation, multi-output saliency prediction, unlabeled auxiliary datasets and channel reduction with teacher assistant supervision. Our approach achieves saliency prediction accuracy on par or better than state-of-the-art methods on DFH1K, UCF-Sports and Hollywood2 benchmarks, while enhancing significantly the efficiency of the model. Code is on https://github.com/feiyanhu/tinyHD