# PitcherNet: Powering the Moneyball Evolution in Baseball Video Analytics

arXiv: https://arxiv.org/abs/2405.07407 · HTML: https://mlsanity.com/paper/2405.07407/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 61 references that are in the map.
- First posted on arXiv 13 May 2024.

## Builds on (most cited first, 24 of 24)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose](https://mlsanity.com/paper/2008.09047/) (2020)
- [MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2111.12707/) (2021)
- [SVFormer: Semi-supervised Video Transformer for Action Recognition](https://mlsanity.com/paper/2211.13222/) (2022)
- [STAR-Transformer: A Spatio-temporal Cross Attention Transformer for Human Action Recognition](https://mlsanity.com/paper/2210.07503/) (2022)
- [Player Tracking and Identification in Ice Hockey](https://mlsanity.com/paper/2110.03090/) (2021)
- [Ice hockey player identification via transformers and weakly supervised learning](https://mlsanity.com/paper/2111.11535/) (2021)
- [Temporal Hockey Action Recognition via Pose and Optical Flows](https://mlsanity.com/paper/1812.09533/) (2018)
- [Jersey Number Recognition using Keyframe Identification from Low-Resolution Broadcast Videos](https://mlsanity.com/paper/2309.06285/) (2023)
- [Player Identification in Hockey Broadcast Videos](https://mlsanity.com/paper/2009.02429/) (2020)
- [Mitigating Motion Blur for Robust 3D Baseball Player Pose Modeling for Pitch Analysis](https://mlsanity.com/paper/2309.01010/) (2023)
- [Distribution and Depth-Aware Transformers for 3D Human Mesh Recovery](https://mlsanity.com/paper/2403.09063/) (2024)
- [Domain-Guided Masked Autoencoders for Unique Player Identification](https://mlsanity.com/paper/2403.11328/) (2024)

## Built on by (most cited first, 4 of 4)

- [DreamPose3D: Hallucinative Diffusion with Prompt Learning for 3D Human Pose Estimation](https://mlsanity.com/paper/2511.09502/) (2025)
- [Avatar4D: Synthesizing Domain-Specific 4D Humans for Real-World Pose Estimation](https://mlsanity.com/paper/2512.16199/) (2025)
- [Scalable Injury-Risk Screening in Baseball Pitching From Broadcast Video](https://mlsanity.com/paper/2603.04864/) (2026)
- [Interpretable Pre-Release Baseball Pitch Type Anticipation from Broadcast 3D Kinematics](https://mlsanity.com/paper/2603.04874/) (2026)

## Abstract (arXiv)

In the high-stakes world of baseball, every nuance of a pitcher's mechanics holds the key to maximizing performance and minimizing runs. Traditional analysis methods often rely on pre-recorded offline numerical data, hindering their application in the dynamic environment of live games. Broadcast video analysis, while seemingly ideal, faces significant challenges due to factors like motion blur and low resolution. To address these challenges, we introduce PitcherNet, an end-to-end automated system that analyzes pitcher kinematics directly from live broadcast video, thereby extracting valuable pitch statistics including velocity, release point, pitch position, and release extension. This system leverages three key components: (1) Player tracking and identification by decoupling actions from player kinematics; (2) Distribution and depth-aware 3D human modeling; and (3) Kinematic-driven pitch statistics. Experimental validation demonstrates that PitcherNet achieves robust analysis results with 96.82% accuracy in pitcher tracklet identification, reduced joint position error by 1.8mm and superior analytics compared to baseline methods. By enabling performance-critical kinematic analysis from broadcast video, PitcherNet paves the way for the future of baseball analytics by optimizing pitching strategies, preventing injuries, and unlocking a deeper understanding of pitcher mechanics, forever transforming the game.