mlsanityData to 7 Oct 2026

Seeking Universal Shot Language Understanding Solutions

arXiv 2603.18448 ↗First posted 19 Mar 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Shot language understanding (SLU) is crucial for cinematic analysis but remains challenging due to its diverse cinematographic dimensions and subjective expert judgment. While vision-language models (VLMs) have shown strong ability in general visual understanding, recent studies reveal judgment discrepancies between VLMs and film experts on SLU tasks. To address this gap, we introduce SLU-SUITE, a comprehensive training and evaluation suite containing 490K human-annotated QA pairs across 33 tasks spanning six film-grounded dimensions. Using SLU-SUITE, we originally observe two insights into VLM-based SLU from: the model side, which diagnoses key bottlenecks of modules; the data side, which quantifies cross-dimensional influences among tasks. These findings motivate our universal SLU solutions from two complementary paradigms: UniShot, a balanced one-for-all generalist trained via dynamic-balanced data mixing, and AgentShots, a prompt-routed expert cluster that maximizes peak dimension performance. Extensive experiments show that our models outperform task-specific ensembles on in-domain tasks and surpass leading commercial VLMs by 22% on out-of-domain tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 47 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

35 of 47 references.

References · 35

  1. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  4. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  5. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  6. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
  7. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  8. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  9. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  10. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  11. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  12. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
23 more
  1. 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretrainingcited by 385
  2. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 359
  3. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  4. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
  5. 2024Mixture of LoRA Expertscited by 179
  6. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 172
  7. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
  8. 2024Towards Modular LLMs by Building and Reusing a Library of LoRAscited by 72
  9. 2024LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wildcited by 68
  10. 2020Object Detection with a Unified Label Space from Multiple Datasetscited by 65
  11. 2024Learning to Route Among Specialized Experts for Zero-Shot Generalizationcited by 65
  12. 2025Towards Understanding Camera Motions in Any Videocited by 62
  13. 2020A Unified Framework for Shot Type Classification Based on Subject Centric Lenscited by 44
  14. 2024Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologiescited by 44
  15. 2019Domain Aggregation Networks for Multi-Source Domain Adaptationcited by 37
  16. 2024MovieBench: A Hierarchical Movie Level Dataset for Long Video Generationcited by 35
  17. 2021MovieCuts: A New Dataset and Benchmark for Cut Type Recognitioncited by 26
  18. 2025CineTechBench: A Benchmark for Cinematographic Technique Understanding and Generationcited by 21
  19. 2020mDALU: Multi-Source Domain Adaptation and Label Unification with Partial Datasetscited by 19
  20. 2022Learning Semantic Segmentation from Multiple Datasets with Label Shiftscited by 14
  21. 2025RefineShot: Rethinking Cinematography Understanding with Foundational Skill Evaluationcited by 10
  22. 2025Stable Cinemetrics : Structured Taxonomy and Evaluation for Professional Video Generationcited by 5
  23. 2024Audio Match Cutting: Finding and Creating Matching Audio Transitions in Movies and Videoscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ShotBench: Expert-Level Cinematic Understanding in Vision-Language Models
  2. 2026CinematicVQA: Benchmarking Film-Grammar Reasoning in Large Vision-Language Models
  3. 2026Scaling Vision-Language Models Is Not Enough to Mitigate Bias
  4. 2025Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model
  5. 2025VLURes: Benchmarking VLM Visual and Linguistic Understanding in Low-Resource Languages
  6. 2024How Well Can Vision Language Models See Image Details?
  7. 2025V2P-Bench: Evaluating Video-Language Understanding with Visual Prompts for Better Human-Model Interaction
  8. 2024Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models