AffectSeek: Agentic Affective Understanding in Long Videos under Vague User Queries
Not on the map. Why?
Abstract (arXiv)
Existing affective understanding studies have mainly focused on recognizing emotions from images, audio signals, or pre-cliped video clips, where the affective evidence is already given. This passive and clip-centered setting does not fully reflect real-world scenarios, in which users often interact with long videos and express their needs through natural-language queries. In this paper, we study Vague-Query-driven video Affective Understanding (VQAU), a new task that requires models to localize affective moments in long videos, predict their emotion categories, and generate evidence-grounded rationales under vague user queries. To support this task, we construct VQAU-Bench, a benchmark that integrates long videos, vague affective queries, temporal clip annotations, emotion labels, and rationale explanations into a unified evaluation framework. VQAU-Bench enables systematic assessment of semantic-temporal-affective alignment, affective moment localization, emotion classification, and rationale generation. To address the multi-step reasoning challenges of VQAU, we further propose AffectSeek, an agentic framework that actively seeks, verifies, and explains affective moments in long videos. AffectSeek decomposes VQAU into intent interpretation, candidate localization, clip verification, emotion reasoning, and rationale generation, and progressively aligns vague user intent with long-video evidence through role-specialized reasoning and cross-stage verification. Experiments show that VQAU remains challenging for existing affective recognition models and single-step vision-language models, while AffectSeek provides a simple yet effective framework for agentic long-video affective understanding.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 41 references that are in the map. First posted on arXiv 7 May 2026.
Builds on
23 of 41 references.
References · 23
- 2017Dense-Captioning Events in Videoscited by 789
- 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
- 2017TALL: Temporal Activity Localization via Language Querycited by 581
- 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
- 2020DFEW: A Large-Scale Database for Recognizing Dynamic Facial Expressions in the Wildcited by 128
- 2024Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuningcited by 117
- 2022FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in Videoscited by 69
- 2017MovieGraphs: Towards Understanding Human-Centric Situations from Videoscited by 67
- 2022MAFW: A Large-scale, Multi-modal, Compound Affective Database for Dynamic Facial Expression Recognition in the Wildcited by 65
- 2025AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Modelscited by 43
- 2024EmoVIT: Revolutionizing Emotion Insights with Visual Instruction Tuningcited by 42
11 more
- 2023MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learningcited by 33
- 2025Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospectscited by 10
- 2024Understanding Emotional Body Expressions via Large Language Modelscited by 8
- 2025From Long Videos to Engaging Clips: A Human-Inspired Video Editing Framework with Multimodal Narrative Understandingcited by 6
- 2024Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Surveycited by 5
- 2026Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understandingcited by 5
- 2023Towards Interpretability in Audio and Visual Affective Machine Learning: A Reviewcited by 3
- 2025Towards Stable Cross-Domain Depression Recognition under Missing Modalitiescited by 3
- 2025Emotion Recognition from Skeleton Data: A Comprehensive Surveycited by 2
- 2023Joint Moment Retrieval and Highlight Detection Via Natural Language Queriescited by 1
- 2025Omni-SILA: Towards Omni-scene Driven Visual Sentiment Identifying, Locating and Attributing in Videoscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Benchmarking Dynamic Affective Reasoning: A Viewer-Centric Video Emotion Dataset
- 2026LongEmo: Towards Emotion Understanding and Reasoning in Long Videos
- 2024StimuVAR: Spatiotemporal Stimuli-aware Video Affective Reasoning with Multimodal Large Language Models
- 2026Adaptive Emotional Video Captioning via Affective Heterogeneous Graph Reasoning and Multi-task Joint Learning
- 2026E3mo-Bench: A Scalable Benchmark for Multimodal Evoked and Expressed Emotion Understanding via Bayesian Pairwise Alignment
- 2025Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models
- 2026AffectVerse: Emotional World Models for Multimodal Affective Computing
- 2026GRACE: Boosting Video MLLMs with Grounded Action-Centric Evidence for Viewer Sentiment Prediction