# SUSiNet: See, Understand and Summarize it

arXiv: https://arxiv.org/abs/1812.00722 · HTML: https://mlsanity.com/paper/1812.00722/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2018.

## Built on by (most cited first, 8 of 8)

- [V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuning](https://mlsanity.com/paper/2404.12353/) (2024)
- [Temporal-Spatial Feature Pyramid for Video Saliency Detection](https://mlsanity.com/paper/2105.04213/) (2021)
- [Multi-modal Segment Assemblage Network for Ad Video Editing with Importance-Coherence Reward](https://mlsanity.com/paper/2209.12164/) (2022)
- [NPF-200: A Multi-Modal Eye Fixation Dataset and Method for Non-Photorealistic Videos](https://mlsanity.com/paper/2308.12163/) (2023)
- [Relevance-guided Audio Visual Fusion for Video Saliency Prediction](https://mlsanity.com/paper/2411.11454/) (2024)
- [Multi-task Learning For Joint Action and Gesture Recognition](https://mlsanity.com/paper/2505.17867/) (2025)
- [Self-Attention Based Generative Adversarial Networks For Unsupervised Video Summarization](https://mlsanity.com/paper/2307.08145/) (2023)
- [Unsupervised Transcript-assisted Video Summarization and Highlight Detection](https://mlsanity.com/paper/2505.23268/) (2025)

## Abstract (arXiv)

In this work we propose a multi-task spatio-temporal network, called SUSiNet, that can jointly tackle the spatio-temporal problems of saliency estimation, action recognition and video summarization. Our approach employs a single network that is jointly end-to-end trained for all tasks with multiple and diverse datasets related to the exploring tasks. The proposed network uses a unified architecture that includes global and task specific layer and produces multiple output types, i.e., saliency maps or classification labels, by employing the same video input. Moreover, one additional contribution is that the proposed network can be deeply supervised through an attention module that is related to human attention as it is expressed by eye-tracking data. From the extensive evaluation, on seven different datasets, we have observed that the multi-task network performs as well as the state-of-the-art single-task methods (or in some cases better), while it requires less computational budget than having one independent network per each task.