# Prompt, Probe, Train, or Annotate? Single-camera sports video understanding in amateur settings

arXiv: https://arxiv.org/abs/2609.28049 · HTML: https://mlsanity.com/paper/2609.28049/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 62 references that are in the map.
- First posted on arXiv 23 Sep 2026.

## Placed in

- World models

## Builds on (most cited first, 17 of 17)

- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [Revisiting Feature Prediction for Learning Visual Representations from Video](https://mlsanity.com/paper/2404.08471/) (2024)
- [SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes](https://mlsanity.com/paper/2304.05170/) (2023)
- [V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning](https://mlsanity.com/paper/2603.14482/) (2026)
- [MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions](https://mlsanity.com/paper/2105.07404/) (2021)
- [SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimap](https://mlsanity.com/paper/2404.11335/) (2024)
- [SportR: A Benchmark for Multimodal Large Language Model Reasoning in Sports](https://mlsanity.com/paper/2511.06499/) (2025)
- [A General Framework for Jersey Number Recognition in Sports Video](https://mlsanity.com/paper/2405.13896/) (2024)
- [What Does Vision Tool-Use Reinforcement Learning Really Learn? Disentangling Tool-Induced and Intrinsic Effects for Crop-and-Zoom](https://mlsanity.com/paper/2602.01334/) (2026)
- [VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification](https://mlsanity.com/paper/2604.01569/) (2026)
- [SoccerNet 2025 Challenges Results](https://mlsanity.com/paper/2508.19182/) (2025)
- [The Hallucination Snowball: Modeling Error Propagation as State Transitions in Multi-Agent LLM Pipelines](https://mlsanity.com/paper/2608.14588/) (2026)
- [Temporal vs. Spatial: Comparing DINOv3 and V-JEPA2 Feature Representations for Video Action Analysis](https://mlsanity.com/paper/2509.21595/) (2025)
- [Towards long-term player tracking with graph hierarchies and domain-specific features](https://mlsanity.com/paper/2502.21242/) (2025)
- [RefereeBench: Are Video MLLMs Ready to be Multi-Sport Referees](https://mlsanity.com/paper/2604.15736/) (2026)
- [SoccerNet 2026 Challenges Results](https://mlsanity.com/paper/2607.07320/) (2026)
- [Towards Comprehensive Basketball Understanding](https://mlsanity.com/paper/2608.23435/) (2026)

## Abstract (arXiv)

Video understanding is usually benchmarked on curated, single-actor, or professionally filmed clips, and a strong score there is routinely read as evidence a model is robust enough for deployment. Amateur team sport is a useful, largely untested place to check that assumption: over eight million students played a school sport in the United States in 2024-25 alone, almost none of it filmed by more than a single fixed camera, with several candidate actors crowded into frame and no operator or second angle to fall back on. Using volleyball as a test case, we ask whether strong performance on general video and world-model benchmarks translates into reliable, per-player attribution once footage is this chaotic, turning footage into statistics through a chain of tasks from finding play boundaries to naming who did what. We evaluate four approaches (prompting and agentic reasoning over frontier vision-language models, classical computer vision with small trained specialists, self-supervised video world models, and manual annotation) at every stage, on 66 amateur matches with 46,648 human-labelled contacts, filmed under conditions no published benchmark uses. No single paradigm wins every stage, and static, single-frame computer vision is not competitive at any stage involving motion or identity. A prompted model segments matches well, yet a far smaller trained model beats it at spotting contacts for a fraction of the cost, and the sport's own rules recover rally outcomes the pixels cannot. Identity is where every automated approach struggles: a jersey number is a static fact temporal reasoning cannot recover if never visible, unlike sporting action, a repeated motor pattern a temporal model can exploit, which is why holistic reasoning improves event detection while identity stays unchanged. We close with where each approach earns its cost, and what transfers beyond volleyball to amateur sport.