# Region-aware Image-based Human Action Retrieval with Transformers

arXiv: https://arxiv.org/abs/2407.09924 · HTML: https://mlsanity.com/paper/2407.09924/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 13 Jul 2024.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [Human Action Recognition and Prediction: A Survey](https://mlsanity.com/paper/1806.11230/) (2018)
- [A Decade Survey of Content Based Image Retrieval using Deep Learning](https://mlsanity.com/paper/2012.00641/) (2020)
- [Neural-Logic Human-Object Interaction Detection](https://mlsanity.com/paper/2311.09817/) (2023)
- [Zero-Shot Action Recognition in Videos: A Survey](https://mlsanity.com/paper/1909.06423/) (2019)
- [Graph Convolution Based Efficient Re-Ranking for Visual Retrieval](https://mlsanity.com/paper/2306.08792/) (2023)
- [Learning Token-based Representation for Image Retrieval](https://mlsanity.com/paper/2112.06159/) (2021)
- [Learning Segment Similarity and Alignment in Large-Scale Content Based Video Retrieval](https://mlsanity.com/paper/2309.11091/) (2023)
- [Learning Efficient Representations for Image-Based Patent Retrieval](https://mlsanity.com/paper/2308.13749/) (2023)
- [Ensembles of Deep Neural Networks for Action Recognition in Still Images](https://mlsanity.com/paper/2003.09893/) (2020)

## Built on by (most cited first, 1 of 1)

- [Dual Conditioned Motion Diffusion for Pose-Based Video Anomaly Detection](https://mlsanity.com/paper/2412.17210/) (2024)

## Abstract (arXiv)

Human action understanding is a fundamental and challenging task in computer vision. Although there exists tremendous research on this area, most works focus on action recognition, while action retrieval has received less attention. In this paper, we focus on the neglected but important task of image-based action retrieval which aims to find images that depict the same action as a query image. We establish benchmarks for this task and set up important baseline methods for fair comparison. We present an end-to-end model that learns rich action representations from three aspects: the anchored person, contextual regions, and the global image. A novel fusion transformer module is designed to model the relationships among different features and effectively fuse them into an action representation. Experiments on the Stanford-40 and PASCAL VOC 2012 Action datasets show that the proposed method significantly outperforms previous approaches for image-based action retrieval.