# A Benchmark for LLM's Understanding of Middle School and High School Science Topics

arXiv: https://arxiv.org/abs/2609.32020 · HTML: https://mlsanity.com/paper/2609.32020/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 46 references that are in the map.
- First posted on arXiv 25 Sep 2026.

## Builds on (most cited first, 30 of 30)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing](https://mlsanity.com/paper/2406.08464/) (2024)
- [2 OLMo 2 Furious](https://mlsanity.com/paper/2501.00656/) (2024)
- [Humans or LLMs as the Judge? A Study on Judgement Biases](https://mlsanity.com/paper/2402.10669/) (2024)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024)
- [LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks](https://mlsanity.com/paper/2406.18403/) (2024)
- [Large Language Models for Education: A Survey and Outlook](https://mlsanity.com/paper/2403.18105/) (2024)
- [A Survey on Data Synthesis and Augmentation for Large Language Models](https://mlsanity.com/paper/2410.12896/) (2024)
- [Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs](https://mlsanity.com/paper/2412.13337/) (2024)
- [LFM2 Technical Report](https://mlsanity.com/paper/2511.23404/) (2025)
- [Large Language Models for Education: A Survey](https://mlsanity.com/paper/2405.13001/) (2024)
- [Survey of different Large Language Model Architectures: Trends, Benchmarks, and Challenges](https://mlsanity.com/paper/2412.03220/) (2024)
- [PhysUniBench: A Multi-Modal Physics Reasoning Benchmark at Undergraduate Level](https://mlsanity.com/paper/2506.17667/) (2025)
- [E-EVAL: A Comprehensive Chinese K-12 Education Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2401.15927/) (2024)
- [Physics Reasoner: Knowledge-Augmented Reasoning for Solving Physics Problems with Large Language Models](https://mlsanity.com/paper/2412.13791/) (2024)
- [LLMs in Education: Novel Perspectives, Challenges, and Opportunities](https://mlsanity.com/paper/2409.11917/) (2024)
- [Dr.Academy: A Benchmark for Evaluating Questioning Capability in Education for Large Language Models](https://mlsanity.com/paper/2408.10947/) (2024)
- [Benchmarking the Pedagogical Knowledge of Large Language Models](https://mlsanity.com/paper/2506.18710/) (2025)
- [PhySense: Principle-Based Physics Reasoning Benchmarking for Large Language Models](https://mlsanity.com/paper/2505.24823/) (2025)
- [ELMES: An Automated Framework for Evaluating Large Language Models in Educational Scenarios](https://mlsanity.com/paper/2507.22947/) (2025)
- [Evaluating LLM-Generated Q&A Test: a Student-Centered Study](https://mlsanity.com/paper/2505.06591/) (2025)
- [Small, Private Language Models as Teammates for Educational Assessment Design](https://mlsanity.com/paper/2605.15015/) (2026)
- [Evaluating the capability of large language models to personalize science texts for diverse middle-school-age learners](https://mlsanity.com/paper/2408.05204/) (2024)
- [TaMPERing with Large Language Models: A Field Guide for using Generative AI in Public Administration Research](https://mlsanity.com/paper/2504.01037/) (2025)
- [Judging the Judges: Human Validation of Multi-LLM Evaluation for High-Quality K--12 Science Instructional Materials](https://mlsanity.com/paper/2602.13243/) (2026)
- [Large Databases Need Small, Open-Weight Language Models](https://mlsanity.com/paper/2606.31808/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly integrated into educational settings, yet educators lack robust, standards-aligned tools to evaluate their effectiveness in K-12 science contexts. Existing benchmarks predominantly assess general language or advanced scientific reasoning, leaving a critical gap in understanding LLMs' performance on content directly relevant to secondary science curricula. To address this gap, we developed a comprehensive NGSS-aligned benchmark for both middle and high school science using a rigorous synthetic data pipeline, multi-judge validation, and item-level psychometric analysis. Nine open-weight LLMs were systematically evaluated using this benchmark, indicating that several smaller, locally deployable models achieved high accuracy across diverse science domains and question types. Our findings indicate that model size did not consistently predict performance, emphasizing the importance of intentional model selection for educational deployment. We then incorporated a human reviewer into the loop, reviewing the items generated by the LLMs for alignment with NGSS standards. The human review indicated that synthetically generated items were not in perfect alignment with the NGSS standards, indicating the benefits of human-in-the-loop item development, the need to explore the intersection of content and pedagogical knowledge, and the need to extend benchmarks to evaluate LLMs' capacity for interactive, evidence-based feedback in educational scenarios.