mlsanityData to 14 Oct 2026

Multi-Granularity Hand Action Detection

arXiv 2306.10858 ↗First posted 19 Jun 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Detecting hand actions in videos is crucial for understanding video content and has diverse real-world applications. Existing approaches often focus on whole-body actions or coarse-grained action categories, lacking fine-grained hand-action localization information. To fill this gap, we introduce the FHA-Kitchens (Fine-Grained Hand Actions in Kitchen Scenes) dataset, providing both coarse- and fine-grained hand action categories along with localization annotations. This dataset comprises 2,377 video clips and 30,047 frames, annotated with approximately 200k bounding boxes and 880 action categories. Evaluation of existing action detection methods on FHA-Kitchens reveals varying generalization capabilities across different granularities. To handle multi-granularity in hand actions, we propose MG-HAD, an End-to-End Multi-Granularity Hand Action Detection method. It incorporates two new designs: Multi-dimensional Action Queries and Coarse-Fine Contrastive Denoising. Extensive experiments demonstrate MG-HAD's effectiveness for multi-granularity hand action detection, highlighting the significance of FHA-Kitchens for future research and real-world applications. The dataset and source code are available at https://github.com/superZ678/MG-HAD.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 53 of its 79 references that are in the map. First posted on arXiv 19 Jun 2023.

Builds on

53 of 79 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2023Segment Anythingcited by 8,234
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  8. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  9. 2017Non-local Neural Networkscited by 2,293
  10. 2018SlowFast Networks for Video Recognitioncited by 1,867
  11. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  12. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
38 more
  1. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  2. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  3. 2021Video Swin Transformercited by 1,091
  4. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,071
  5. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
  6. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  7. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  8. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  9. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 686
  10. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 637
  11. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 573
  12. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 537
  13. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
  14. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
  15. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 488
  16. 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 484
  17. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
  18. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  19. 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
  20. 2018A Short Note about Kinetics-600cited by 333
  21. 2017Temporal Relational Reasoning in Videoscited by 324
  22. 2018Video Action Transformer Networkcited by 316
  23. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 310
  24. 2023Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistlescited by 290
  25. 2017Temporal Segment Networks for Action Recognition in Videoscited by 279
  26. 2020FineGym: A Hierarchical Video Dataset for Fine-grained Action Understandingcited by 234
  27. 2020Multi-Modal Domain Adaptation for Fine-Grained Action Recognitioncited by 115
  28. 2020Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Thingscited by 111
  29. 2020A Short Note on the Kinetics-700-2020 Human Action Datasetcited by 92
  30. 2020Actor-Context-Actor Relation Network for Spatio-Temporal Action Localizationcited by 91
  31. 2020The AVA-Kinetics Localized Human Actions Video Datasetcited by 82
  32. 2023Dense Distinct Query for End-to-End Object Detectioncited by 77
  33. 2020Asynchronous Interaction Aggregation for Action Detectioncited by 65
  34. 2022ViTPose++: Vision Transformer for Generic Body Pose Estimationcited by 65
  35. 2022ReAct: Temporal Action Detection with Relational Queriescited by 48
  36. 2021Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognitioncited by 29
  37. 2017Temporal 3D ConvNets: New Architecture and Transfer Learning for Video Classificationcited by 27
  38. 2023STMixer: A One-Stage Sparse Action Detectorcited by 22

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.