# TSalV360: A Method and Dataset for Text-driven Saliency Detection in 360-Degrees Videos

arXiv: https://arxiv.org/abs/2509.26208 · HTML: https://mlsanity.com/paper/2509.26208/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 28 of 28)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [Tangent Images for Mitigating Spherical Distortion](https://mlsanity.com/paper/1912.09390/) (2019)
- [Motion Guided Attention for Video Salient Object Detection](https://mlsanity.com/paper/1909.07061/) (2019)
- [Cube Padding for Weakly-Supervised Saliency Prediction in 360deg Videos](https://mlsanity.com/paper/1806.01320/) (2018)
- [ViNet: Pushing the limits of Visual Modality for Audio-Visual Saliency Prediction](https://mlsanity.com/paper/2012.06170/) (2020)
- [Predicting Head Movement in Panoramic Video: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1710.10755/) (2017)
- [A Memory Network Approach for Story-based Temporal Summarization of 360deg Videos](https://mlsanity.com/paper/1805.02838/) (2018)
- [Towards Accurate Text-based Image Captioning with Content Diversity Exploration](https://mlsanity.com/paper/2105.03236/) (2021)
- [Pano2Vid: Automatic Cinematography for Watching 360circ Videos](https://mlsanity.com/paper/1612.02335/) (2016)
- [Deep 360 Pilot: Learning a Deep Agent for Piloting through 360deg Sports Video](https://mlsanity.com/paper/1705.01759/) (2017)
- [ATSal: An Attention Based Architecture for Saliency Prediction in 360 Videos](https://mlsanity.com/paper/2011.10600/) (2020)
- [A Deep Ranking Model for Spatio-Temporal Highlight Detection from a 360 Video](https://mlsanity.com/paper/1801.10312/) (2018)
- [Panoramic Vision Transformer for Saliency Detection in 360deg Videos](https://mlsanity.com/paper/2209.08956/) (2022)
- [Making 360circ Video Watchable in 2D: Learning Videography for Click Free Viewing](https://mlsanity.com/paper/1703.00495/) (2017)
- [Video Salient Object Detection via Contrastive Features and Attention Modules](https://mlsanity.com/paper/2111.02368/) (2021)
- [Query-based Video Summarization with Pseudo Label Supervision](https://mlsanity.com/paper/2307.01945/) (2023)
- [DiffSal: Joint Audio and Video Learning for Diffusion Saliency Prediction](https://mlsanity.com/paper/2403.01226/) (2024)
- [Video Saliency Prediction Using Enhanced Spatiotemporal Alignment Network](https://mlsanity.com/paper/2001.00292/) (2020)
- [360VOTS: Visual Object Tracking and Segmentation in Omnidirectional Videos](https://mlsanity.com/paper/2404.13953/) (2024)
- [How is Visual Attention Influenced by Text Guidance? Database and Model](https://mlsanity.com/paper/2404.07537/) (2024)
- [Text-Audio-Visual-conditioned Diffusion Model for Video Saliency Prediction](https://mlsanity.com/paper/2504.14267/) (2025)
- [SD-VSum: A Method and Dataset for Script-Driven Video Summarization](https://mlsanity.com/paper/2505.03319/) (2025)
- [Understanding Video Scenes through Text: Insights from Text-based Video Question Answering](https://mlsanity.com/paper/2309.01380/) (2023)
- [An Integrated System for Spatio-Temporal Summarization of 360-degrees Videos](https://mlsanity.com/paper/2312.02576/) (2023)

## Abstract (arXiv)

In this paper, we deal with the task of text-driven saliency detection in 360-degrees videos. For this, we introduce the TSV360 dataset which includes 16,000 triplets of ERP frames, textual descriptions of salient objects/events in these frames, and the associated ground-truth saliency maps. Following, we extend and adapt a SOTA visual-based approach for 360-degrees video saliency detection, and develop the TSalV360 method that takes into account a user-provided text description of the desired objects and/or events. This method leverages a SOTA vision-language model for data representation and integrates a similarity estimation module and a viewport spatio-temporal cross-attention mechanism, to discover dependencies between the different data modalities. Quantitative and qualitative evaluations using the TSV360 dataset, showed the competitiveness of TSalV360 compared to a SOTA visual-based approach and documented its competency to perform customized text-driven saliency detection in 360-degrees videos.