mlsanityData to 7 Oct 2026

ActionSwitch: Class-agnostic Detection of Simultaneous Actions in Streaming Videos

arXiv 2407.12987 ↗First posted 17 Jul 2024cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Online Temporal Action Localization (On-TAL) is a critical task that aims to instantaneously identify action instances in untrimmed streaming videos as soon as an action concludes -- a major leap from frame-based Online Action Detection (OAD). Yet, the challenge of detecting overlapping actions is often overlooked even though it is a common scenario in streaming videos. Current methods that can address concurrent actions depend heavily on class information, limiting their flexibility. This paper introduces ActionSwitch, the first class-agnostic On-TAL framework capable of detecting overlapping actions. By obviating the reliance on class information, ActionSwitch provides wider applicability to various situations, including overlapping actions of the same class or scenarios where class information is unavailable. This approach is complemented by the proposed "conservativeness loss", which directly embeds a conservative decision-making principle into the loss function for On-TAL. Our ActionSwitch achieves state-of-the-art performance in complex datasets, including Epic-Kitchens 100 targeting the challenging egocentric view and FineAction consisting of fine-grained actions.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 46 of its 56 references that are in the map. First posted on arXiv 17 Jul 2024.

Builds on

46 of 56 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020End-to-End Object Detection with Transformerscited by 7,384
  3. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  4. 2015Fast R-CNNcited by 2,930
  5. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  6. 2018SlowFast Networks for Video Recognitioncited by 1,862
  7. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
  8. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  9. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 899
  10. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  11. 2016R-FCN: Object Detection via Region-based Fully Convolutional Networkscited by 414
  12. 2017Soft-NMS -- Improving Object Detection With One Line of Codecited by 380
34 more
  1. 2019BMN: Boundary-Matching Network for Temporal Action Proposal Generationcited by 327
  2. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  3. 2018BSN: Boundary Sensitive Network for Temporal Action Proposal Generationcited by 271
  4. 2019G-TAD: Sub-Graph Localization for Temporal Action Detectioncited by 248
  5. 2019Graph Convolutional Networks for Temporal Action Localizationcited by 208
  6. 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 178
  7. 2021End-to-end Temporal Action Detection with Transformercited by 150
  8. 2023CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matchingcited by 133
  9. 2021Relaxed Transformer Decoders for Direct Action Proposal Generationcited by 116
  10. 2021Learning Open-World Object Proposals without Learning to Classifycited by 109
  11. 2015Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videoscited by 93
  12. 2021Long Short-Term Transformer for Online Action Detectioncited by 87
  13. 2021Class-agnostic Object Detection with Multi-modal Transformercited by 85
  14. 2021OadTR: Online Action Detection with Transformerscited by 83
  15. 2018Temporal Recurrent Networks for Online Action Detectioncited by 79
  16. 2017RED: Reinforced Encoder-Decoder Networks for Action Anticipationcited by 77
  17. 2021DETReg: Unsupervised Pretraining with Region Priors for Object Detectioncited by 76
  18. 2019Grounding Human-to-Vehicle Advice for Self-driving Vehiclescited by 71
  19. 2016Online Action Detectioncited by 68
  20. 2020BSN++: Complementary Boundary Regressor with Scale-Balanced Relation Modeling for Temporal Action Proposal Generationcited by 65
  21. 2021Generic Event Boundary Detection: A Benchmark for Event Segmentationcited by 64
  22. 2021FineAction: A Fine-Grained Video Dataset for Temporal Action Localizationcited by 60
  23. 2022Real-time Online Video Detection with Temporal Smoothing Transformerscited by 58
  24. 2020Multi-shot Temporal Event Localization: a Benchmarkcited by 52
  25. 2022Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinitycited by 41
  26. 2019Learning to Discriminate Information for Online Action Detectioncited by 37
  27. 2022GateHUB: Gated History Unit with Background Suppression for Online Action Detectioncited by 27
  28. 2019StartNet: Online Detection of Action Start in Untrimmed Videoscited by 21
  29. 2022Colar: Effective and Efficient Online Action Detection by Consulting Exemplarscited by 21
  30. 2020WOAD: Weakly Supervised Online Action Detection in Untrimmed Videoscited by 19
  31. 2018Online Detection of Action Start in Untrimmed, Streaming Videoscited by 17
  32. 2020Class-agnostic Object Detectioncited by 17
  33. 2021Temporally smooth online action detection using cycle-consistent future anticipationcited by 13
  34. 2022SimOn: A Simple Framework for Online Temporal Action Localizationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Open-ended Hierarchical Streaming Video Understanding with Vision Language Modelscited by 6
  2. 2026OnPoint: Offline-to-Online Multi-Level Distillation for Point-Supervised Online Temporal Action Localizationcited by 1
  3. 2026OZ-TAL: Online Zero-Shot Temporal Action Localizationcited by 0
  4. 2026When Should the Count Change? Learning State Maintenance for Causal Video Countingcited by 0