# RacketVision: A Multiple Racket Sports Benchmark for Unified Ball and Racket Analysis

arXiv: https://arxiv.org/abs/2511.17045 · HTML: https://mlsanity.com/paper/2511.17045/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 21 Nov 2025.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [RTMDet: An Empirical Study of Designing Real-Time Object Detectors](https://mlsanity.com/paper/2212.07784/) (2022)
- [RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose](https://mlsanity.com/paper/2303.07399/) (2023)
- [SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos](https://mlsanity.com/paper/2011.13367/) (2020)
- [SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos](https://mlsanity.com/paper/2204.06918/) (2022)
- [TacticAI: an AI assistant for football tactics](https://mlsanity.com/paper/2310.10553/) (2023)
- [TrackNet: A Deep Learning Network for Tracking High-speed and Tiny Objects in Sports Applications](https://mlsanity.com/paper/1907.03698/) (2019)
- [Achieving Human Level Competitive Robot Table Tennis](https://mlsanity.com/paper/2408.03906/) (2024)
- [Semi-Supervised Training to Improve Player and Ball Detection in Soccer](https://mlsanity.com/paper/2204.06859/) (2022)
- [Blur Interpolation Transformer for Real-World Motion from Blur](https://mlsanity.com/paper/2211.11423/) (2022)
- [Learning coordinated badminton skills for legged manipulators](https://mlsanity.com/paper/2505.22974/) (2025)
- [Animation from Blur: Multi-modal Blur Decomposition with Motion Guidance](https://mlsanity.com/paper/2207.10123/) (2022)
- [DeepBall: Deep Neural-Network Ball Detector](https://mlsanity.com/paper/1902.07304/) (2019)
- [Within the Dynamic Context: Inertia-aware 3D Human Modeling with Pose Sequence](https://mlsanity.com/paper/2403.19160/) (2024)
- [Shape from Blur: Recovering Textured 3D Shape and Motion of Fast Moving Objects](https://mlsanity.com/paper/2106.08762/) (2021)
- [ToMiE: Towards Explicit Exoskeleton for the Reconstruction of Complicated 3D Human Avatars](https://mlsanity.com/paper/2410.08082/) (2024)
- [Sequential Gaussian Avatars with Hierarchical Motion Context](https://mlsanity.com/paper/2411.16768/) (2024)
- [Table Tennis Stroke Detection and Recognition Using Ball Trajectory Data](https://mlsanity.com/paper/2302.09657/) (2023)
- [LATTE-MV: Learning to Anticipate Table Tennis Hits from Monocular Videos](https://mlsanity.com/paper/2503.20936/) (2025)
- [SGA-INTERACT: A 3D Skeleton-based Benchmark for Group Activity Understanding in Modern Basketball Tactic](https://mlsanity.com/paper/2503.06522/) (2025)
- [Towards Ball Spin and Trajectory Analysis in Table Tennis Broadcast Videos via Physically Grounded Synthetic-to-Real Transfer](https://mlsanity.com/paper/2504.19863/) (2025)
- [Learnable SMPLify: A Neural Solution for Optimization-Free Human Pose Inverse Kinematics](https://mlsanity.com/paper/2508.13562/) (2025)

## Built on by (most cited first, 2 of 2)

- [Stepping VLMs onto the Court: Benchmarking Spatial Intelligence in Sports](https://mlsanity.com/paper/2603.09896/) (2026)
- [P-SRM: Selective Recovery of Rejected Predictions in Visual Tracking](https://mlsanity.com/paper/2609.39832/) (2026)

## Abstract (arXiv)

We introduce RacketVision, a novel dataset and benchmark for advancing computer vision in sports analytics, covering table tennis, tennis, and badminton. The dataset is the first to provide large-scale, fine-grained annotations for racket pose alongside traditional ball positions, enabling research into complex human-object interactions. It is designed to tackle three interconnected tasks: fine-grained ball tracking, articulated racket pose estimation, and predictive ball trajectory forecasting. Our evaluation of established baselines reveals a critical insight for multi-modal fusion: while naively concatenating racket pose features degrades performance, a CrossAttention mechanism is essential to unlock their value, leading to trajectory prediction results that surpass strong unimodal baselines. RacketVision provides a versatile resource and a strong starting point for future research in dynamic object tracking, conditional motion forecasting, and multimodal analysis in sports. Project page at https://github.com/OrcustD/RacketVision