mlsanityData to 7 Oct 2026

Streaming Detection of Queried Event Start

arXiv 2412.03567 ↗First posted 4 Dec 2024cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Robotics, autonomous driving, augmented reality, and many embodied computer vision applications must quickly react to user-defined events unfolding in real time. We address this setting by proposing a novel task for multimodal video understanding-Streaming Detection of Queried Event Start (SDQES). The goal of SDQES is to identify the beginning of a complex event as described by a natural language query, with high accuracy and low latency. We introduce a new benchmark based on the Ego4D dataset, as well as new task-specific metrics to study streaming multimodal detection of diverse events in an egocentric video setting. Inspired by parameter-efficient fine-tuning methods in NLP and for video tasks, we propose adapter-based baselines that enable image-to-video transfer learning, allowing for efficient online video modeling. We evaluate three vision-language backbones and three adapter architectures on both short-clip and untrimmed video settings.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 51 of its 74 references that are in the map. First posted on arXiv 4 Dec 2024.

Builds on

51 of 74 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  5. 2021ViViT: A Video Vision Transformercited by 1,610
  6. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  7. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  8. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  9. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  10. 2017Dense-Captioning Events in Videoscited by 789
  11. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  12. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
38 more
  1. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  2. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
  3. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
  4. 2017TALL: Temporal Activity Localization via Language Querycited by 581
  5. 2020Rescaling Egocentric Visioncited by 529
  6. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  7. 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 506
  8. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  9. 2017Temporal Action Detection with Structured Segment Networkscited by 265
  10. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 234
  11. 2022Egocentric Video-Language Pretrainingcited by 231
  12. 2020The EPIC-KITCHENS Dataset: Collection, Challenges and Baselinescited by 221
  13. 2016The THUMOS Challenge on Action Recognition for Videos "in the Wild"cited by 213
  14. 2022Learning Video Representations from Large Language Modelscited by 204
  15. 2021Anticipative Video Transformercited by 197
  16. 2018Rethinking the Faster R-CNN Architecture for Temporal Action Localizationcited by 194
  17. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  18. 2022Frozen CLIP Models are Efficient Video Learnerscited by 181
  19. 2022Revisiting the "Video" in Video-Language Understandingcited by 167
  20. 2017R-C3D: Region Convolutional 3D Network for Temporal Activity Detectioncited by 161
  21. 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 161
  22. 2020Rolling-Unrolling LSTMs for Action Anticipation from First-Person Videocited by 119
  23. 2020EGO-TOPO: Environment Affordances from Egocentric Videocited by 111
  24. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  25. 2016Quasi-Recurrent Neural Networkscited by 87
  26. 2018Localizing Moments in Video with Temporal Languagecited by 83
  27. 2018Temporal Recurrent Networks for Online Action Detectioncited by 79
  28. 2017RED: Reinforced Encoder-Decoder Networks for Action Anticipationcited by 77
  29. 2020What is More Likely to Happen Next? Video-and-Language Future Event Predictioncited by 61
  30. 2016End-to-end Concept Word Detection for Video Captioning, Retrieval, and Question Answeringcited by 60
  31. 2021Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Modelscited by 52
  32. 2023NaQ: Leveraging Narrations as Queries to Supervise Episodic Memorycited by 46
  33. 2024EgoVideo: Exploring Egocentric Foundation Model and Downstream Adaptationcited by 44
  34. 2016First-Person Activity Forecasting with Online Inverse Reinforcement Learningcited by 40
  35. 2023Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learningcited by 31
  36. 2024Time-, Memory- and Parameter-Efficient Visual Adaptationcited by 28
  37. 2019StartNet: Online Detection of Action Start in Untrimmed Videoscited by 21
  38. 2023GroundNLQ @ Ego4D Natural Language Queries Challenge 2023cited by 20

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Eyes Wide Open: Ego Proactive Video-LLM for Streaming Videocited by 25
  2. 2025DiscussLLM: Teaching Large Language Models When to Speakcited by 1
  3. 2026Linear Scaling Video VLMs for Long Video Understandingcited by 1