# SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines

arXiv: https://arxiv.org/abs/2502.14739 · HTML: https://mlsanity.com/paper/2502.14739/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2025.

## Built on by (most cited first, 30 of 79)

- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [Scaling Latent Reasoning via Looped Language Models](https://mlsanity.com/paper/2510.25741/) (2025)
- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning](https://mlsanity.com/paper/2506.10521/) (2025)
- [Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning](https://mlsanity.com/paper/2504.13941/) (2025)
- [Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-Thought](https://mlsanity.com/paper/2505.15431/) (2025)
- [Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning](https://mlsanity.com/paper/2509.25300/) (2025)
- [InternBootcamp: Boosting LLM Reasoning with Verifiable Task Scaling](https://mlsanity.com/paper/2508.08636/) (2025)
- [Trace Length is a Simple Uncertainty Signal in Reasoning Models](https://mlsanity.com/paper/2510.10409/) (2025)
- [InnovatorBench: Evaluating Agents' Ability to Conduct Innovative LLM Research](https://mlsanity.com/paper/2510.27598/) (2025)
- [MTR-Bench: A Comprehensive Benchmark for Multi-Turn Reasoning Evaluation](https://mlsanity.com/paper/2505.17123/) (2025)
- [Adaptive Deep Reasoning: Triggering Deep Thinking When Needed](https://mlsanity.com/paper/2505.20101/) (2025)
- [Thinking Out Loud: Do Reasoning Models Know When They're Right?](https://mlsanity.com/paper/2504.06564/) (2025)
- [Agentic Proposing: Enhancing Large Language Model Reasoning via Compositional Skill Synthesis](https://mlsanity.com/paper/2602.03279/) (2026)
- [Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning](https://mlsanity.com/paper/2509.21193/) (2025)
- [Omni-Thinker: Scaling Multi-Task RL in LLMs with Hybrid Reward and Task Scheduling](https://mlsanity.com/paper/2507.14783/) (2025)
- [VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Training](https://mlsanity.com/paper/2508.03058/) (2025)
- [SUPERChem: A Multimodal Reasoning Benchmark in Chemistry](https://mlsanity.com/paper/2512.01274/) (2025)
- [DNR Bench: Benchmarking Over-Reasoning in Reasoning LLMs](https://mlsanity.com/paper/2503.15793/) (2025)
- [Reasoning Curriculum: Bootstrapping Broad LLM Reasoning from Math](https://mlsanity.com/paper/2510.26143/) (2025)
- [RelayLLM: Efficient Reasoning via Collaborative Decoding](https://mlsanity.com/paper/2601.05167/) (2026)
- [How to Fine-Tune a Reasoning Model? A Teacher-Student Cooperation Framework to Synthesize Student-Consistent SFT Data](https://mlsanity.com/paper/2604.14164/) (2026)
- [Reported Confidence in LLMs Tracks Commitment More Than Correctness](https://mlsanity.com/paper/2606.29490/) (2026)
- [Nondeterministic Polynomial-time Problem Challenge: An Ever-Scaling Reasoning Benchmark for LLMs](https://mlsanity.com/paper/2504.11239/) (2025)
- [KORGym: A Dynamic Game Platform for LLM Reasoning Evaluation](https://mlsanity.com/paper/2505.14552/) (2025)
- [REA-RL: Reflection-Aware Online Reinforcement Learning for Efficient Reasoning](https://mlsanity.com/paper/2505.19862/) (2025)
- [Truly Assessing Fluid Intelligence of Large Language Models through Dynamic Reasoning Evaluation](https://mlsanity.com/paper/2506.02648/) (2025)
- [WildSci: Advancing Scientific Reasoning from In-the-Wild Literature](https://mlsanity.com/paper/2601.05567/) (2026)
- [DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution](https://mlsanity.com/paper/2601.13761/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated remarkable proficiency in mainstream academic disciplines such as mathematics, physics, and computer science. However, human knowledge encompasses over 200 specialized disciplines, far exceeding the scope of existing benchmarks. The capabilities of LLMs in many of these specialized fields-particularly in light industry, agriculture, and service-oriented disciplines-remain inadequately evaluated. To address this gap, we present SuperGPQA, a comprehensive benchmark that evaluates graduate-level knowledge and reasoning capabilities across 285 disciplines. Our benchmark employs a novel Human-LLM collaborative filtering mechanism to eliminate trivial or ambiguous questions through iterative refinement based on both LLM responses and expert feedback. Our experimental results reveal significant room for improvement in the performance of current state-of-the-art LLMs across diverse knowledge domains (e.g., the reasoning-focused model DeepSeek-R1 achieved the highest accuracy of 61.82% on SuperGPQA), highlighting the considerable gap between current model capabilities and artificial general intelligence. Additionally, we present comprehensive insights from our management of a large-scale annotation process, involving over 80 expert annotators and an interactive Human-LLM collaborative system, offering valuable methodological guidance for future research initiatives of comparable scope.