mlsanityData to 7 Oct 2026

Query matching for spatio-temporal action detection with query-based object detector

arXiv 2409.18408 ↗First posted 27 Sep 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a method that extends the query-based object detection model, DETR, to spatio-temporal action detection, which requires maintaining temporal consistency in videos. Our proposed method applies DETR to each frame and uses feature shift to incorporate temporal information. However, DETR's object queries in each frame may correspond to different objects, making a simple feature shift ineffective. To overcome this issue, we propose query matching across different frames, ensuring that queries for the same object are matched and used for the feature shift. Experimental results show that performance on the JHMDB21 dataset improves significantly when query features are shifted using the proposed query matching.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 29 references that are in the map. First posted on arXiv 27 Sep 2024.

Builds on

24 of 29 references.

References · 24

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  6. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  7. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 494
  8. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  9. 2021Conditional DETR for Fast Training Convergencecited by 422
  10. 2021Efficient DETR: Improving End-to-End Object Detector with Dense Priorcited by 128
  11. 2019A Survey of Deep Learning-based Object Detectioncited by 106
  12. 2020Actor-Context-Actor Relation Network for Spatio-Temporal Action Localizationcited by 91
12 more
  1. 2018Actor-Centric Relation Networkcited by 87
  2. 2022AdaMixer: A Fast-Converging Query-Based Object Detectorcited by 78
  3. 2017Action Tubelet Detector for Spatio-Temporal Action Localizationcited by 77
  4. 2021Token Shift Transformer for Video Classificationcited by 61
  5. 2019You Only Watch Once: A Unified CNN Architecture for Real-Time Spatiotemporal Action Localizationcited by 58
  6. 2021TubeR: Tubelet Transformer for Video Action Detectioncited by 55
  7. 2014Finding Action Tubescited by 50
  8. 2023STMixer: A One-Stage Sparse Action Detectorcited by 22
  9. 2023End-to-End Spatio-Temporal Action Localisation with Video Transformerscited by 20
  10. 2019Spatio-temporal Action Recognition: A Surveycited by 5
  11. 2022Vision Transformer with Cross-attention by Temporal Shift for Efficient Action Recognitioncited by 5
  12. 2023DOAD: Decoupled One Stage Action Detection Networkcited by 1

Built on by

Not cited yet.