# Vid2Player: Controllable Video Sprites that Behave and Appear like Professional Tennis Players

arXiv: https://arxiv.org/abs/2008.04524 · HTML: https://mlsanity.com/paper/2008.04524/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2020.

## Built on by (most cited first, 30 of 30)

- [Playable Video Generation](https://mlsanity.com/paper/2101.12195/) (2021)
- [S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling](https://mlsanity.com/paper/2101.06571/) (2021)
- [Multi-modal 3D Human Pose Estimation with 2D Weak Supervision in Autonomous Driving](https://mlsanity.com/paper/2112.12141/) (2021)
- [Spotting Temporally Precise, Fine-Grained Events in Video](https://mlsanity.com/paper/2207.10213/) (2022)
- [Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognition](https://mlsanity.com/paper/2109.01305/) (2021)
- [Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Video](https://mlsanity.com/paper/2404.09833/) (2024)
- [MonoTrack: Shuttle trajectory reconstruction from monocular badminton video](https://mlsanity.com/paper/2204.01899/) (2022)
- [Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with Whitted-Style Ray Tracing](https://mlsanity.com/paper/2308.03280/) (2023)
- [T-DEED: Temporal-Discriminability Enhancer Encoder-Decoder for Precise Event Spotting in Sports Videos](https://mlsanity.com/paper/2404.05392/) (2024)
- [Audio-driven Neural Gesture Reenactment with Video Motion Graphs](https://mlsanity.com/paper/2207.11524/) (2022)
- [Neural Volumetric Reconstruction for Coherent Synthetic Aperture Sonar](https://mlsanity.com/paper/2306.09909/) (2023)
- [ASTRA: An Action Spotting TRAnsformer for Soccer Videos](https://mlsanity.com/paper/2404.01891/) (2024)
- [Strumming to the Beat: Audio-Conditioned Contrastive Video Textures](https://mlsanity.com/paper/2104.02687/) (2021)
- [Domain Adaptive 3D Pose Augmentation for In-the-wild Human Mesh Recovery](https://mlsanity.com/paper/2206.10457/) (2022)
- [Pose Magic: Efficient and Temporally Consistent Human Pose Estimation with a Hybrid Mamba-GCN Network](https://mlsanity.com/paper/2408.02922/) (2024)
- [A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision](https://mlsanity.com/paper/2307.03353/) (2023)
- [Relit-NeuLF: Efficient Relighting and Novel View Synthesis via Neural 4D Light Field](https://mlsanity.com/paper/2310.14642/) (2023)
- [ViSTec: Video Modeling for Sports Technique Recognition and Tactical Analysis](https://mlsanity.com/paper/2402.15952/) (2024)
- [Precise Event Spotting in Sports Videos: Solving Long-Range Dependency and Class Imbalance](https://mlsanity.com/paper/2503.00147/) (2025)
- [Graph-based 3D Human Pose Estimation using WiFi Signals](https://mlsanity.com/paper/2511.19105/) (2025)
- [Sketch-based Creativity Support Tools using Deep Learning](https://mlsanity.com/paper/2111.09991/) (2021)
- [Animating Street View](https://mlsanity.com/paper/2310.08534/) (2023)
- [Deep Learning for Sports Video Event Detection: Tasks, Datasets, Methods, and Challenges](https://mlsanity.com/paper/2505.03991/) (2025)
- [AdaSpot: Spend Resolution Where It Matters for Precise Event Spotting](https://mlsanity.com/paper/2602.22073/) (2026)
- [ESCAPE: Energy-based Selective Adaptive Correction for Out-of-distribution 3D Human Pose Estimation](https://mlsanity.com/paper/2407.14605/) (2024)
- [Multi-Focus Temporal Shifting for Precise Event Spotting in Sports Videos](https://mlsanity.com/paper/2507.07381/) (2025)
- [Visual Intelligence through Human Interaction](https://mlsanity.com/paper/2111.06913/) (2021)
- [Towards Precise Action Spotting: Addressing Temporal Misalignment in Labels with Dynamic Label Assignment](https://mlsanity.com/paper/2504.00149/) (2025)
- [VirtualFencer: Generating Fencing Bouts based on Strategies Extracted from In-the-Wild Videos](https://mlsanity.com/paper/2507.00261/) (2025)
- [A Survey of Large Models in Sports](https://mlsanity.com/paper/2608.14377/) (2026)

## Abstract (arXiv)

We present a system that converts annotated broadcast video of tennis matches into interactively controllable video sprites that behave and appear like professional tennis players. Our approach is based on controllable video textures, and utilizes domain knowledge of the cyclic structure of tennis rallies to place clip transitions and accept control inputs at key decision-making moments of point play. Most importantly, we use points from the video collection to model a player's court positioning and shot selection decisions during points. We use these behavioral models to select video clips that reflect actions the real-life player is likely to take in a given match play situation, yielding sprites that behave realistically at the macro level of full points, not just individual tennis motions. Our system can generate novel points between professional tennis players that resemble Wimbledon broadcasts, enabling new experiences such as the creation of matchups between players that have not competed in real life, or interactive control of players in the Wimbledon final. According to expert tennis players, the rallies generated using our approach are significantly more realistic in terms of player behavior than video sprite methods that only consider the quality of motion transitions during video synthesis.