# Actions as Moving Points

arXiv: https://arxiv.org/abs/2001.04608 · HTML: https://mlsanity.com/paper/2001.04608/

## Key facts

- Cited by 55 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 14 Jan 2020.

## Built on by (most cited first, 30 of 55)

- [Relaxed Transformer Decoders for Direct Action Proposal Generation](https://mlsanity.com/paper/2102.01894/) (2021)
- [Negative Sample Matters: A Renaissance of Metric Learning for Temporal Grounding](https://mlsanity.com/paper/2109.04872/) (2021)
- [MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions](https://mlsanity.com/paper/2105.07404/) (2021)
- [TubeR: Tubelet Transformer for Video Action Detection](https://mlsanity.com/paper/2104.00969/) (2021)
- [ROAD: The ROad event Awareness Dataset for Autonomous Driving](https://mlsanity.com/paper/2102.11585/) (2021)
- [Target Adaptive Context Aggregation for Video Scene Graph Generation](https://mlsanity.com/paper/2108.08121/) (2021)
- [RGB-Event Fusion for Moving Object Detection in Autonomous Driving](https://mlsanity.com/paper/2209.08323/) (2022)
- [Unified Graph Structured Models for Video Understanding](https://mlsanity.com/paper/2103.15662/) (2021)
- [Holistic Interaction Transformer Network for Action Detection](https://mlsanity.com/paper/2210.12686/) (2022)
- [PointTAD: Multi-Label Temporal Action Detection with Learnable Query Points](https://mlsanity.com/paper/2210.11035/) (2022)
- [STMixer: A One-Stage Sparse Action Detector](https://mlsanity.com/paper/2303.15879/) (2023)
- [End-to-End Spatio-Temporal Action Localisation with Video Transformers](https://mlsanity.com/paper/2304.12160/) (2023)
- [Unified Keypoint-based Action Recognition Framework via Structured Keypoint Pooling](https://mlsanity.com/paper/2303.15270/) (2023)
- [Efficient Video Action Detection with Token Dropout and Context Refinement](https://mlsanity.com/paper/2304.08451/) (2023)
- [Dual DETRs for Multi-Label Temporal Action Detection](https://mlsanity.com/paper/2404.00653/) (2024)
- [End-to-End Semi-Supervised Learning for Video Action Detection](https://mlsanity.com/paper/2203.04251/) (2022)
- [Spatio-Temporal Action Detection Under Large Motion](https://mlsanity.com/paper/2209.02250/) (2022)
- [Temporal Perceiver: A General Architecture for Arbitrary Boundary Detection](https://mlsanity.com/paper/2203.00307/) (2022)
- [Tubelet-Contrastive Self-Supervision for Video-Efficient Generalization](https://mlsanity.com/paper/2303.11003/) (2023)
- [Semi-supervised Active Learning for Video Action Detection](https://mlsanity.com/paper/2312.07169/) (2023)
- [Computer Vision for Primate Behavior Analysis in the Wild](https://mlsanity.com/paper/2401.16424/) (2024)
- [Survey of Action Recognition, Spotting and Spatio-Temporal Localization in Soccer -- Current Trends and Research Perspectives](https://mlsanity.com/paper/2309.12067/) (2023)
- [ACDnet: An action detection network for real-time edge computing based on flow-guided feature approximation and memory aggregation](https://mlsanity.com/paper/2102.13493/) (2021)
- [Video-based cattle identification and action recognition](https://mlsanity.com/paper/2110.07103/) (2021)
- [YOWOv2: A Stronger yet Efficient Multi-level Detection Framework for Real-time Spatio-temporal Action Detection](https://mlsanity.com/paper/2302.06848/) (2023)
- [MINOTAUR: Multi-task Video Grounding From Multimodal Queries](https://mlsanity.com/paper/2302.08063/) (2023)
- [On Occlusions in Video Action Detection: Benchmark Datasets And Training Recipes](https://mlsanity.com/paper/2410.19553/) (2024)
- [Stable Mean Teacher for Semi-supervised Video Action Detection](https://mlsanity.com/paper/2412.07072/) (2024)
- [Game State and Spatio-temporal Action Detection in Soccer using Graph Neural Networks and 3D Convolutional Networks](https://mlsanity.com/paper/2502.15462/) (2025)
- [A Survey on Deep Learning-based Spatio-temporal Action Detection](https://mlsanity.com/paper/2308.01618/) (2023)

## Abstract (arXiv)

The existing action tubelet detectors often depend on heuristic anchor design and placement, which might be computationally expensive and sub-optimal for precise localization. In this paper, we present a conceptually simple, computationally efficient, and more precise action tubelet detection framework, termed as MovingCenter Detector (MOC-detector), by treating an action instance as a trajectory of moving points. Based on the insight that movement information could simplify and assist action tubelet detection, our MOC-detector is composed of three crucial head branches: (1) Center Branch for instance center detection and action recognition, (2) Movement Branch for movement estimation at adjacent frames to form trajectories of moving points, (3) Box Branch for spatial extent detection by directly regressing bounding box size at each estimated center. These three branches work together to generate the tubelet detection results, which could be further linked to yield video-level tubes with a matching strategy. Our MOC-detector outperforms the existing state-of-the-art methods for both metrics of frame-mAP and video-mAP on the JHMDB and UCF101-24 datasets. The performance gap is more evident for higher video IoU, demonstrating that our MOC-detector is particularly effective for more precise action detection. We provide the code at https://github.com/MCG-NJU/MOC-Detector.