TRINITY: A Multi-Perspective Benchmark for Personal-Style Video Highlight Detection
Not on the map. Why?
Abstract (arXiv)
Traditional video highlight detection relies on a narrow, event-centric definition of saliency, which often fails to generalize to unconstrained personal videos where highlights are heterogeneous and perspective-dependent. To address this, we introduce TRINITY, a multi-perspective benchmark that decomposes highlight saliency into three complementary dimensions, Event, Emotion, and Nature, within a unified temporal framework. Leveraging this multi-faceted view, we propose a shared-backbone multi-branch architecture designed for parallel multi-perspective prediction via view-specific experts. Comprehensive experiments demonstrate that our method significantly outperforms state-of-the-art baselines, achieving gains of +7.15/+3.62 mAP (rho=15%/50%) on Mr. HiSum and +10.82 mAP on YouTube Highlights. These results validate that multi-perspective modeling provides a more robust and comprehensive formulation of video saliency, especially for complex real-world scenarios. The benchmark and relevant codes will be released upon acceptance. The benchmark is available at https://huggingface.co/datasets/vanilladucky/TRINITY and the code is available at https://github.com/vanilladucky/TRINITY.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 58 references that are in the map. First posted on arXiv 30 Aug 2026.
Builds on
35 of 58 references.
References · 35
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
- 2020Gradient Surgery for Multi-Task Learningcited by 1,256
- 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
- 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
- 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
- 2017Dense-Captioning Events in Videoscited by 793
- 2017Localizing Moments in Video with Natural Languagecited by 663
- 2017TALL: Temporal Activity Localization via Language Querycited by 584
23 more
- 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
- 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 255
- 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 190
- 2023Query-Dependent Video Representation for Moment Retrieval and Highlight Detectioncited by 166
- 2022UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detectioncited by 148
- 2023Hierarchical Video-Moment Retrieval and Step-Captioningcited by 83
- 2023Knowing Where to Focus: Event-aware Transformer for Video Groundingcited by 79
- 2023Independent Component Alignment for Multi-Task Learningcited by 70
- 2024TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detectioncited by 59
- 2014Coherent Multi-Sentence Video Description with Variable Level of Detailcited by 45
- 2016Video2GIF: Automatic Generation of Animated GIFs from Videocited by 41
- 2021Cross-category Video Highlight Detection via Set-based Learningcited by 38
- 2024CSTA: CNN-based Spatiotemporal Attention for Video Summarizationcited by 23
- 2018PHD-GIFs: Personalized Highlight Detection for Automatic GIF Creationcited by 19
- 2024COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Trainingcited by 18
- 2022Learning Pixel-Level Distinctions for Video Highlight Detectioncited by 16
- 2017Deep Asymmetric Multi-task Feature Learningcited by 15
- 2023VEATIC: Video-based Emotion and Affect Tracking in Context Datasetcited by 12
- 2024Unsupervised Modality-Transferable Video Highlight Detection with Representation Activation Sequence Learningcited by 7
- 2024Unsupervised Video Highlight Detection by Learning from Audio and Visual Recurrencecited by 6
- 2025SummDiff: Generative Modeling of Video Summarization with Diffusioncited by 4
- 2025HierSum: A Global and Local Attention Mechanism for Video Summarizationcited by 3
- 2025AHA -- Predicting What Matters Next: Online Highlight Detection Without Looking Aheadcited by 2
Built on by
Not cited yet.