mlsanityData to 7 Oct 2026

Action tube generation by person query matching for spatio-temporal action detection

arXiv 2503.12969 ↗First posted 17 Mar 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper proposes a method for spatio-temporal action detection (STAD) that directly generates action tubes from the original video without relying on post-processing steps such as IoU-based linking and clip splitting. Our approach applies query-based detection (DETR) to each frame and matches DETR queries to link the same person across frames. We introduce the Query Matching Module (QMM), which uses metric learning to bring queries for the same person closer together across frames compared to queries for different people. Action classes are predicted using the sequence of queries obtained from QMM matching, allowing for variable-length inputs from videos longer than a single clip. Experimental results on JHMDB, UCF101-24, and AVA datasets demonstrate that our method performs well for large position changes of people while offering superior computational efficiency and lower resource requirements.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 47 references that are in the map. First posted on arXiv 17 Mar 2025.

Builds on

41 of 47 references.

References · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2017Mask R-CNNcited by 7,259
  5. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  6. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  7. 2015Fast R-CNNcited by 2,930
  8. 2018YOLOv3: An Incremental Improvementcited by 2,652
  9. 2017The Kinetics Human Action Video Datasetcited by 1,894
  10. 2018SlowFast Networks for Video Recognitioncited by 1,862
  11. 2021ViViT: A Video Vision Transformercited by 1,610
  12. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
29 more
  1. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  2. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
  3. 2016Simple Online and Realtime Trackingcited by 862
  4. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  5. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 494
  6. 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 489
  7. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  8. 2019Bag of Tricks and A Strong Baseline for Deep Person Re-identificationcited by 429
  9. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 379
  10. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
  11. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 273
  12. 2018Long-Term Feature Banks for Detailed Video Understandingcited by 239
  13. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  14. 2022TubeDETR: Spatio-Temporal Video Grounding with Transformerscited by 97
  15. 2020Actor-Context-Actor Relation Network for Spatio-Temporal Action Localizationcited by 91
  16. 2018Actor-Centric Relation Networkcited by 87
  17. 2017Action Tubelet Detector for Spatio-Temporal Action Localizationcited by 77
  18. 2019You Only Watch Once: A Unified CNN Architecture for Real-Time Spatiotemporal Action Localizationcited by 58
  19. 2016Online Real-time Multiple Spatiotemporal Action Localisation and Predictioncited by 57
  20. 2020Actions as Moving Pointscited by 55
  21. 2021TubeR: Tubelet Transformer for Video Action Detectioncited by 55
  22. 2019STEP: Spatio-Temporal Progressive Learning for Video Action Detectioncited by 51
  23. 2014Finding Action Tubescited by 50
  24. 2017Tube Convolutional Neural Network (T-CNN) for Action Detection in Videoscited by 50
  25. 2015Learning to track for spatio-temporal action localizationcited by 44
  26. 2016Deep Learning for Detecting Multiple Space-Time Action Tubes in Videoscited by 35
  27. 2023End-to-End Spatio-Temporal Action Localisation with Video Transformerscited by 20
  28. 2023Efficient Video Action Detection with Token Dropout and Context Refinementcited by 18
  29. 2022Spatio-Temporal Action Detection Under Large Motioncited by 14

Built on by

Not cited yet.