mlsanityData to 14 Oct 2026

Shot2Story: A New Benchmark for Comprehensive Understanding of Multi-shot Videos

arXiv 2312.10300 ↗First posted 16 Dec 2023cs.CVcited by 54

Not on the map. Why?

Abstract (arXiv)

A short clip of video may contain progression of multiple events and an interesting story line. A human need to capture both the event in every shot and associate them together to understand the story behind it. In this work, we present a new multi-shot video understanding benchmark Shot2Story with detailed shot-level captions, comprehensive video summaries and question-answering pairs. To facilitate better semantic understanding of videos, we provide captions for both visual signals and human narrations. We design several distinct tasks including single-shot video captioning, multi-shot video summarization, and multi-shot video question answering. Preliminary experiments show some challenges to generate a long and comprehensive video summary for multi-shot videos. Nevertheless, the generated imperfect summaries can already achieve competitive performance on existing video understanding tasks such as video question-answering, promoting an under-explored setting of video understanding with detailed summaries.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Dec 2023.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 11; 2025: 31; 2026: 11.

54 papers.

Sort:

By area:

  1. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  2. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 309
  3. 2023Video Understanding with Large Language Models: A Surveycited by 206
  4. 2024LongVLM: Efficient Long Video Understanding via Large Language Modelscited by 147
  5. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 140
  6. 2025Scaling RL to Long Videoscited by 84
  7. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 64
  8. 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 63
  9. 2025OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contextscited by 51
  10. 2024VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Formatcited by 47
  11. 2024V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuningcited by 46
  12. 2024VideoCLIP-XL: Advancing Long Description Understanding for Video CLIP Modelscited by 41
38 more
  1. 2025CausalVQA: A Physically Grounded Causal Reasoning Benchmark for Video Modelscited by 35
  2. 2025Mavors: Multi-granularity Video Representation for Multimodal Large Language Modelcited by 32
  3. 2025VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinkingcited by 30
  4. 2025CineTrans: Learning to Generate Videos with Cinematic Transitions via Masked Diffusion Modelscited by 29
  5. 2025ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Modelscited by 27
  6. 2025AVoCaDO: An Audiovisual Video Captioner Driven by Temporal Orchestrationcited by 27
  7. 2025Advancing Complex Video Object Segmentation via Progressive Concept Constructioncited by 26
  8. 2025Streaming Video Instruction Tuningcited by 26
  9. 2025LiveStar: Live Streaming Assistant for Real-World Online Video Understandingcited by 25
  10. 2025T2AV-Compass: Towards Unified Evaluation for Text-to-Audio-Video Generationcited by 24
  11. 2024RoomTour3D: Geometry-Aware Video-Instruction Tuning for Embodied Navigationcited by 23
  12. 2025Knowing Your Target: Target-Aware Transformer Makes Better Spatio-Temporal Video Groundingcited by 18
  13. 2024MLP Can Be A Good Transformer Learnercited by 17
  14. 2025COCONut-PanCap: Joint Panoptic Segmentation and Grounded Captions for Fine-Grained Understanding and Generationcited by 12
  15. 2025What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentationscited by 11
  16. 2026TimeChat-Captioner: Scripting Multi-Scene Videos with Time-Aware and Structural Audio-Visual Captionscited by 11
  17. 2025MVU-Eval: Towards Multi-Video Understanding Evaluation for Multimodal LLMscited by 10
  18. 2025AuroraLong: Bringing RNNs Back to Efficient Open-Ended Video Understandingcited by 9
  19. 2026MuSS: A Large-Scale Dataset and Cinematic Narrative Benchmark for Multi-Shot Subject-to-Video Generationcited by 9
  20. 20254D-Bench: Benchmarking Multi-modal Large Language Models for 4D Object Understandingcited by 8
  21. 2025Text-to-Edit: Controllable End-to-End Video Ad Creation via Multimodal LLMscited by 7
  22. 2025An LMM for Efficient Video Understanding via Reinforced Compression of Video Cubescited by 7
  23. 2025WAVE: Learning Unified & Versatile Audio-Visual Embeddings with Multimodal LLMcited by 7
  24. 2025VDC-Agent: When Video Detailed Captioners Evolve Themselves via Agentic Self-Reflectioncited by 6
  25. 2025ChronusOmni: Improving Time Awareness of Omni Large Language Modelscited by 6
  26. 2025Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challengescited by 5
  27. 2025CineVerse: Consistent Keyframe Synthesis for Cinematic Scene Compositioncited by 4
  28. 2026AutoCut: End-to-end advertisement video editing based on multimodal discretization and controllable generationcited by 4
  29. 2026Harnessing Streaming Video in the Wildcited by 4
  30. 2025HierSum: A Global and Local Attention Mechanism for Video Summarizationcited by 3
  31. 2024Teaching Human Behavior Improves Content Understanding Abilities Of LLMscited by 2
  32. 2025AHA -- Predicting What Matters Next: Online Highlight Detection Without Looking Aheadcited by 2
  33. 2025NOAH: Benchmarking Narrative Prior driven Hallucination and Omission in Video Large Language Modelscited by 2
  34. 2026LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streamscited by 2
  35. 2024Shotluck Holmes: A Family of Efficient Small-Scale Large Language Vision Models For Video Captioning and Summarizationcited by 1
  36. 2025Can Video Large Multimodal Models Think Like Doubters-or Double-Down: A Study on Defeasible Video Entailmentcited by 1
  37. 2026MVEB: Massive Video Embedding Benchmarkcited by 1
  38. 2026MSEditor: Toward Consistent Multi-Shot Video Editingcited by 1

50 most cited of 54 All 54 (JSON)