# Large Language Models Achieve Gold Medal Performance at the International Olympiad on Astronomy & Astrophysics (IOAA)

arXiv: https://arxiv.org/abs/2510.05016 · HTML: https://mlsanity.com/paper/2510.05016/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 6 Oct 2025.

## Builds on (most cited first, 12 of 12)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Accelerating scientific discovery with Co-Scientist](https://mlsanity.com/paper/2502.18864/) (2025)
- [Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models](https://mlsanity.com/paper/2406.09403/) (2024)
- [CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs](https://mlsanity.com/paper/2406.18521/) (2024)
- [Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Proving](https://mlsanity.com/paper/2502.07640/) (2025)
- [Competitive Programming with Large Reasoning Models](https://mlsanity.com/paper/2502.06807/) (2025)
- [Effective Training Data Synthesis for Improving MLLM Chart Understanding](https://mlsanity.com/paper/2508.06492/) (2025)
- [Physics Supernova: AI Agent Matches Elite Gold Medalists at IPhO 2025](https://mlsanity.com/paper/2509.01659/) (2025)
- [A Deep Learning Approach for Active Anomaly Detection of Extragalactic Transients](https://mlsanity.com/paper/2103.12102/) (2021)
- [Automated Algorithmic Discovery for Scientific Computing through LLM-Guided Evolutionary Search: A Case Study in Gravitational-Wave Detection](https://mlsanity.com/paper/2508.03661/) (2025)
- [AstroMLab 1: Who Wins Astronomy Jeopardy!?](https://mlsanity.com/paper/2407.11194/) (2024)
- [AstroMLab 4: Benchmark-Topping Performance in Astronomy Q&A with a 70B-Parameter Domain-Specialized Reasoning Model](https://mlsanity.com/paper/2505.17592/) (2025)

## Built on by (most cited first, 3 of 3)

- [What Understanding Means in AI-Laden Astronomy](https://mlsanity.com/paper/2601.10038/) (2026)
- [amc: The Automated Mission Classifier for Telescope Bibliographies](https://mlsanity.com/paper/2512.11202/) (2025)
- [Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branches](https://mlsanity.com/paper/2606.01145/) (2026)

## Abstract (arXiv)

While task-specific demonstrations show early success in applying large language models (LLMs) to automate some astronomical research tasks, they only provide incomplete views of all necessary capabilities in solving astronomy problems, calling for more thorough understanding of LLMs' strengths and limitations. So far, existing benchmarks and evaluations focus on simple question-answering that primarily tests astronomical knowledge and fails to evaluate the complex reasoning required for real-world research in the discipline. Here, we address this gap by systematically benchmarking five state-of-the-art LLMs on the International Olympiad on Astronomy and Astrophysics (IOAA) exams, which are designed to examine deep conceptual understanding, multi-step derivations, and multimodal analysis. With average scores of 85.6% and 84.2%, Gemini 2.5 Pro and GPT-5 (the two top-performing models) not only achieve gold medal level performance but also rank in the top two among ~200-300 participants in all four IOAA theory exams evaluated (2022-2025). In comparison, results on the data analysis exams show more divergence. GPT-5 still excels in the exams with an 88.5% average score, ranking top 10 among the participants in the four most recent IOAAs, while other models' performances drop to 48-76%. Furthermore, our in-depth error analysis underscores conceptual reasoning, geometric reasoning, and spatial visualization (52-79% accuracy) as consistent weaknesses among all LLMs. Hence, although LLMs approach peak human performance in theory exams, critical gaps must be addressed before they can serve as autonomous research agents in astronomy.