# Generating and Evaluating Tests for K-12 Students with Language Model Simulations: A Case Study on Sentence Reading Efficiency

arXiv: https://arxiv.org/abs/2310.06837 · HTML: https://mlsanity.com/paper/2310.06837/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 63 references that are in the map.
- First posted on arXiv 10 Oct 2023.

## Builds on (most cited first, 17 of 17)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [8-bit Optimizers via Block-wise Quantization](https://mlsanity.com/paper/2110.02861/) (2021)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [Knowledge Tracing: A Survey](https://mlsanity.com/paper/2201.06953/) (2022)
- [Fine-Tuning Language Models with Advantage-Induced Policy Alignment](https://mlsanity.com/paper/2306.02231/) (2023)
- [Supervised and Unsupervised Neural Approaches to Text Readability](https://mlsanity.com/paper/1907.11779/) (2019)
- [Question Generation for Adaptive Education](https://mlsanity.com/paper/2106.04262/) (2021)
- [Automatic Generation of Word Problems for Academic Education via Natural Language Processing (NLP)](https://mlsanity.com/paper/2109.13123/) (2021)

## Built on by (most cited first, 16 of 16)

- [Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation](https://mlsanity.com/paper/2402.04616/) (2024)
- [SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction](https://mlsanity.com/paper/2507.05129/) (2025)
- [Psychometric Alignment: Capturing Human Knowledge Distributions via Language Models](https://mlsanity.com/paper/2407.15645/) (2024)
- [Evaluating and Optimizing Educational Content with Large Language Model Judgments](https://mlsanity.com/paper/2403.02795/) (2024)
- [Smart Agent-Based Modeling: On the Use of Large Language Models in Computer Simulations](https://mlsanity.com/paper/2311.06330/) (2023)
- [IterAlign: Iterative Constitutional Alignment of Large Language Models](https://mlsanity.com/paper/2403.18341/) (2024)
- [RED: Unleashing Token-Level Rewards from Holistic Feedback via Reward Redistribution](https://mlsanity.com/paper/2411.08302/) (2024)
- [The Promises and Pitfalls of Using Language Models to Measure Instruction Quality in Education](https://mlsanity.com/paper/2404.02444/) (2024)
- [Take Out Your Calculators: Estimating the Real Difficulty of Question Items with LLM Student Simulations](https://mlsanity.com/paper/2601.09953/) (2026)
- [Can LLMs Reliably Simulate Real Students' Abilities in Mathematics and Reading Comprehension?](https://mlsanity.com/paper/2507.08232/) (2025)
- [COGENT: A Curriculum-oriented Framework for Generating Grade-appropriate Educational Content](https://mlsanity.com/paper/2506.09367/) (2025)
- [Automatic Generation of Inference Making Questions for Reading Comprehension Assessments](https://mlsanity.com/paper/2506.08260/) (2025)
- [Reconstructing Item Characteristic Curves using Fine-Tuned Large Language Models](https://mlsanity.com/paper/2601.02580/) (2026)
- [Who Annotates in NLP? A Large-scale Assessment of Human Annotation Reporting between 2018 and 2025](https://mlsanity.com/paper/2606.02255/) (2026)
- ["Act Like a 5th Grader" is Not Enough: Bounding Knowledge in LLM-Based User Simulators](https://mlsanity.com/paper/2608.30033/) (2026)
- [A Unified Response Framework for Questions and Tutoring Dialogues](https://mlsanity.com/paper/2610.19532/) (2026)

## Abstract (arXiv)

Developing an educational test can be expensive and time-consuming, as each item must be written by experts and then evaluated by collecting hundreds of student responses. Moreover, many tests require multiple distinct sets of questions administered throughout the school year to closely monitor students' progress, known as parallel tests. In this study, we focus on tests of silent sentence reading efficiency, used to assess students' reading ability over time. To generate high-quality parallel tests, we propose to fine-tune large language models (LLMs) to simulate how previous students would have responded to unseen items. With these simulated responses, we can estimate each item's difficulty and ambiguity. We first use GPT-4 to generate new test items following a list of expert-developed rules and then apply a fine-tuned LLM to filter the items based on criteria from psychological measurements. We also propose an optimal-transport-inspired technique for generating parallel tests and show the generated tests closely correspond to the original test's difficulty and reliability based on crowdworker responses. Our evaluation of a generated test with 234 students from grades 2 to 8 produces test scores highly correlated (r=0.93) to those of a standard test form written by human experts and evaluated across thousands of K-12 students.