# SportQA: A Benchmark for Sports Understanding in Large Language Models

arXiv: https://arxiv.org/abs/2402.15862 · HTML: https://mlsanity.com/paper/2402.15862/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 24 Feb 2024.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [The Web as a Knowledge-base for Answering Complex Questions](https://mlsanity.com/paper/1803.06643/) (2018)
- [Is GPT-3 a Good Data Annotator?](https://mlsanity.com/paper/2212.10450/) (2022)
- [Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework](https://mlsanity.com/paper/2305.03268/) (2023)
- [Reasoning Implicit Sentiment with Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.11255/) (2023)
- [KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base](https://mlsanity.com/paper/2007.03875/) (2020)
- [Few-shot In-context Learning for Knowledge Base Question Answering](https://mlsanity.com/paper/2305.01750/) (2023)
- [Quasar: Datasets for Question Answering by Search and Reading](https://mlsanity.com/paper/1707.03904/) (2017)
- [LiveQA: A Question Answering Dataset over Sports Live](https://mlsanity.com/paper/2010.00526/) (2020)
- [Integrating Physiological Time Series and Clinical Notes with Transformer for Early Prediction of Sepsis](https://mlsanity.com/paper/2203.14469/) (2022)
- [Sporthesia: Augmenting Sports Videos Using Natural Language](https://mlsanity.com/paper/2209.03434/) (2022)
- [Combining Machine Learning and Human Experts to Predict Match Outcomes in Football: A Baseline Model](https://mlsanity.com/paper/2012.04380/) (2020)
- [Empirical Quantitative Analysis of COVID-19 Forecasting Models](https://mlsanity.com/paper/2110.00174/) (2021)
- [VREN: Volleyball Rally Dataset with Expression Notation Language](https://mlsanity.com/paper/2209.13846/) (2022)
- [Universal Information Extraction with Meta-Pretrained Self-Retrieval](https://mlsanity.com/paper/2306.10444/) (2023)
- [Graph Encoding and Neural Network Approaches for Volleyball Analytics: From Game Outcome to Individual Play Predictions](https://mlsanity.com/paper/2308.11142/) (2023)

## Built on by (most cited first, 15 of 15)

- [BIG-Bench Extra Hard](https://mlsanity.com/paper/2502.19187/) (2025)
- [SPORTU: A Comprehensive Sports Understanding Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2410.08474/) (2024)
- [Towards Universal Soccer Video Understanding](https://mlsanity.com/paper/2412.01820/) (2024)
- [RUPBench: Benchmarking Reasoning Under Perturbations for Robustness Evaluation in Large Language Models](https://mlsanity.com/paper/2406.11020/) (2024)
- [SoccerMaster: A Vision Foundation Model for Soccer Understanding](https://mlsanity.com/paper/2512.11016/) (2025)
- [Domain Adaptation of VLM for Soccer Video Understanding](https://mlsanity.com/paper/2505.13860/) (2025)
- [FSBench: A Figure Skating Benchmark for Advancing Artistic Sports Understanding](https://mlsanity.com/paper/2504.19514/) (2025)
- [Towards Evaluating Proactive Risk Awareness of Multimodal Language Models](https://mlsanity.com/paper/2505.17455/) (2025)
- [DeepSport: A Multimodal Large Language Model for Comprehensive Sports Video Reasoning via Agentic Reinforcement Learning](https://mlsanity.com/paper/2511.12908/) (2025)
- [SV3.3B: A Sports Video Understanding Model for Action Recognition](https://mlsanity.com/paper/2507.17844/) (2025)
- [TennisTV: Do Multimodal Large Language Models Understand Tennis Rallies?](https://mlsanity.com/paper/2509.15602/) (2025)
- [Stepping VLMs onto the Court: Benchmarking Spatial Intelligence in Sports](https://mlsanity.com/paper/2603.09896/) (2026)
- [AI for Handball: predicting and explaining the 2024 Olympic Games tournament with Deep Learning and Large Language Models](https://mlsanity.com/paper/2407.15987/) (2024)
- [Towards Temporal Compositional Reasoning in Long-Form Sports Videos](https://mlsanity.com/paper/2604.22226/) (2026)
- [Evaluating point-light biological motion in multimodal large language models](https://mlsanity.com/paper/2509.23517/) (2025)

## Abstract (arXiv)

A deep understanding of sports, a field rich in strategic and dynamic content, is crucial for advancing Natural Language Processing (NLP). This holds particular significance in the context of evaluating and advancing Large Language Models (LLMs), given the existing gap in specialized benchmarks. To bridge this gap, we introduce SportQA, a novel benchmark specifically designed for evaluating LLMs in the context of sports understanding. SportQA encompasses over 70,000 multiple-choice questions across three distinct difficulty levels, each targeting different aspects of sports knowledge from basic historical facts to intricate, scenario-based reasoning tasks. We conducted a thorough evaluation of prevalent LLMs, mainly utilizing few-shot learning paradigms supplemented by chain-of-thought (CoT) prompting. Our results reveal that while LLMs exhibit competent performance in basic sports knowledge, they struggle with more complex, scenario-based sports reasoning, lagging behind human expertise. The introduction of SportQA marks a significant step forward in NLP, offering a tool for assessing and enhancing sports understanding in LLMs.