mlsanityData to 14 Oct 2026

ProTAL: A Drag-and-Link Video Programming Framework for Temporal Action Localization

arXiv 2505.17555 ↗First posted 23 May 2025cs.HC cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Temporal Action Localization (TAL) aims to detect the start and end timestamps of actions in a video. However, the training of TAL models requires a substantial amount of manually annotated data. Data programming is an efficient method to create training labels with a series of human-defined labeling functions. However, its application in TAL faces difficulties of defining complex actions in the context of temporal video frames. In this paper, we propose ProTAL, a drag-and-link video programming framework for TAL. ProTAL enables users to define key events by dragging nodes representing body parts and objects and linking them to constrain the relations (direction, distance, etc.). These definitions are used to generate action labels for large-scale unlabelled videos. A semi-supervised method is then employed to train TAL models with such labels. We demonstrate the effectiveness of ProTAL through a usage scenario and a user study, providing insights into designing video programming framework.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 79 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

43 of 79 references.

References · 43

  1. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  2. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  3. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  4. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  5. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  6. 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 372
  7. 2017Temporal Segment Networks for Action Recognition in Videoscited by 279
  8. 2017Temporal Action Detection with Structured Segment Networkscited by 267
  9. 2021Revisiting Skeleton-based Action Recognitioncited by 257
  10. 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
  11. 2020FineGym: A Hierarchical Video Dataset for Fine-grained Action Understandingcited by 234
  12. 2019Graph Convolutional Networks for Temporal Action Localizationcited by 208
31 more
  1. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  2. 2018Rethinking the Faster R-CNN Architecture for Temporal Action Localizationcited by 194
  3. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  4. 2023RTMPose: Real-Time Multi-Person Pose Estimation based on MMPosecited by 175
  5. 2023TriDet: Temporal Action Detection with Relative Boundary Modelingcited by 142
  6. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 135
  7. 2017UntrimmedNets for Weakly Supervised Action Recognition and Detectioncited by 130
  8. 2020Learning Human-Object Interaction Detection using Interaction Pointscited by 108
  9. 2018Training Complex Models with Multi-Task Weak Supervisioncited by 106
  10. 2020Video Self-Stitching Graph Network for Temporal Action Localizationcited by 104
  11. 2017Weakly Supervised Action Localization by Sparse Temporal Pooling Networkcited by 100
  12. 2020Actor-Context-Actor Relation Network for Spatio-Temporal Action Localizationcited by 91
  13. 2020SF-Net: Single-Frame Supervision for Temporal Action Localizationcited by 62
  14. 2019Weakly-supervised Action Localization with Background Modelingcited by 53
  15. 2017Learning the Structure of Generative Models without Labeled Datacited by 50
  16. 2021Temporal Action Segmentation from Timestamp Supervisioncited by 48
  17. 2023End-to-End Temporal Action Detection with 1B Parameters Across 1000 Framescited by 47
  18. 2017Cascaded Boundary Regression for Temporal Action Detectioncited by 35
  19. 2019Learning Dependency Structures for Weak Supervision Modelscited by 32
  20. 2021Graph Convolutional Module for Temporal Action Localization in Videoscited by 29
  21. 2022Holistic Interaction Transformer Network for Action Detectioncited by 28
  22. 2021The Blessings of Unlabeled Background in Untrimmed Videoscited by 16
  23. 2023Temporal Action Localization with Enhanced Instant Discriminabilitycited by 13
  24. 2022Semi-Supervised Temporal Action Detection with Proposal-Free Maskingcited by 10
  25. 2023VideoPro: A Visual Analytics Approach for Interactive Video Programmingcited by 7
  26. 2020Subspace Clustering for Action Recognition with Covariance Representations and Temporal Pruningcited by 6
  27. 2021Relation Modeling in Spatio-Temporal Action Localizationcited by 5
  28. 2021VideoModerator: A Risk-aware Framework for Multimodal Video Moderation in E-Commercecited by 5
  29. 2021TagRuler: Interactive Tool for Span-Level Data Programming by Demonstrationcited by 4
  30. 2021EventAnchor: Reducing Human Interactions in Event Annotation of Racket Sports Videoscited by 3
  31. 2019Animated Drag and Drop Interaction for Dynamic Multidimensional Graphscited by 1

Built on by

Not cited yet.