# PoseRAC: Pose Saliency Transformer for Repetitive Action Counting

arXiv: https://arxiv.org/abs/2303.08450 · HTML: https://mlsanity.com/paper/2303.08450/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 15 Mar 2023.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [TAM: Temporal Adaptive Module for Video Recognition](https://mlsanity.com/paper/2005.06803/) (2020)
- [TransPose: Keypoint Localization via Transformer](https://mlsanity.com/paper/2012.14214/) (2020)
- [BlazePose: On-device Real-time Body Pose tracking](https://mlsanity.com/paper/2006.10204/) (2020)
- [TokenPose: Learning Keypoint Tokens for Human Pose Estimation](https://mlsanity.com/paper/2104.03516/) (2021)
- [Counting Out Time: Class Agnostic Video Repetition Counting in the Wild](https://mlsanity.com/paper/2006.15418/) (2020)
- [TransRAC: Encoding Multi-scale Temporal Correlation with Transformers for Repetitive Action Counting](https://mlsanity.com/paper/2204.01018/) (2022)
- [Context-aware and Scale-insensitive Temporal Repetition Counting](https://mlsanity.com/paper/2005.08465/) (2020)
- [Real-World Repetition Estimation by Div, Grad and Curl](https://mlsanity.com/paper/1802.09971/) (2018)
- [M3ST: Mix at Three Levels for Speech Translation](https://mlsanity.com/paper/2212.03657/) (2022)
- [Structure from Recurrent Motion: From Rigidity to Recurrency](https://mlsanity.com/paper/1804.06510/) (2018)
- [SSVMR: Saliency-based Self-training for Video-Music Retrieval](https://mlsanity.com/paper/2302.09328/) (2023)
- [(Quasi)Periodicity Quantification in Video Data, Using Topology](https://mlsanity.com/paper/1704.08382/) (2017)
- [A Dynamic Graph Interactive Framework with Label-Semantic Injection for Spoken Language Understanding](https://mlsanity.com/paper/2211.04023/) (2022)
- [Bi-directional Object-context Prioritization Learning for Saliency Ranking](https://mlsanity.com/paper/2203.09416/) (2022)

## Built on by (most cited first, 17 of 17)

- [Human Motion Instruction Tuning](https://mlsanity.com/paper/2411.16805/) (2024)
- [SyncFusion: Multimodal Onset-synchronized Video-to-Audio Foley Synthesis](https://mlsanity.com/paper/2310.15247/) (2023)
- [CountLLM: Towards Generalizable Repetitive Action Counting via Large Language Model](https://mlsanity.com/paper/2503.17690/) (2025)
- [OVR: A Dataset for Open Vocabulary Temporal Repetition Counting in Videos](https://mlsanity.com/paper/2407.17085/) (2024)
- [Every Shot Counts: Using Exemplars for Repetition Counting in Videos](https://mlsanity.com/paper/2403.18074/) (2024)
- [Out-of-Distributed Semantic Pruning for Robust Semi-Supervised Learning](https://mlsanity.com/paper/2305.18158/) (2023)
- [HumMUSS: Human Motion Understanding using State Space Models](https://mlsanity.com/paper/2404.10880/) (2024)
- [Full Resolution Repetition Counting](https://mlsanity.com/paper/2305.13778/) (2023)
- [Advancements in Repetitive Action Counting: Joint-Based PoseRAC Model With Improved Performance](https://mlsanity.com/paper/2308.08632/) (2023)
- [FCA-RAC: First Cycle Annotated Repetitive Action Counting](https://mlsanity.com/paper/2406.12178/) (2024)
- [MultiCounter: Multiple Action Agnostic Repetition Counting in Untrimmed Videos](https://mlsanity.com/paper/2409.04035/) (2024)
- [PushupBench: Your VLM is not good at counting pushups](https://mlsanity.com/paper/2604.23407/) (2026)
- [I2KD-SLU: An Intra-Inter Knowledge Distillation Framework for Zero-Shot Cross-Lingual Spoken Language Understanding](https://mlsanity.com/paper/2310.02594/) (2023)
- [IVAC-P2L: Leveraging Irregular Repetition Priors for Improving Video Action Counting](https://mlsanity.com/paper/2403.11959/) (2024)
- [GMFL-Net: A Global Multi-geometric Feature Learning Network for Repetitive Action Counting](https://mlsanity.com/paper/2409.00330/) (2024)
- [Unsupervised Discovery of Long-Term Spatiotemporal Periodic Workflows in Human Activities](https://mlsanity.com/paper/2511.14945/) (2025)
- [Every Step of the Way: Video-based Parkinsonian Turning Step Counting](https://mlsanity.com/paper/2606.27918/) (2026)

## Abstract (arXiv)

This paper presents a significant contribution to the field of repetitive action counting through the introduction of a new approach called Pose Saliency Representation. The proposed method efficiently represents each action using only two salient poses instead of redundant frames, which significantly reduces the computational cost while improving the performance. Moreover, we introduce a pose-level method, PoseRAC, which is based on this representation and achieves state-of-the-art performance on two new version datasets by using Pose Saliency Annotation to annotate salient poses for training. Our lightweight model is highly efficient, requiring only 20 minutes for training on a GPU, and infers nearly 10x faster compared to previous methods. In addition, our approach achieves a substantial improvement over the previous state-of-the-art TransRAC, achieving an OBO metric of 0.56 compared to 0.29 of TransRAC. The code and new dataset are available at https://github.com/MiracleDance/PoseRAC for further research and experimentation, making our proposed approach highly accessible to the research community.