# Automated Scoring for Reading Comprehension via In-context BERT Tuning

arXiv: https://arxiv.org/abs/2205.09864 · HTML: https://mlsanity.com/paper/2205.09864/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 31 references that are in the map.
- First posted on arXiv 19 May 2022.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [MetaICL: Learning to Learn In Context](https://mlsanity.com/paper/2110.15943/) (2021)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Meta-learning via Language Model In-context Tuning](https://mlsanity.com/paper/2110.07814/) (2021)
- [NeuSpell: A Neural Spelling Correction Toolkit](https://mlsanity.com/paper/2010.11085/) (2020)
- [Should College Dropout Prediction Models Include Protected Attributes?](https://mlsanity.com/paper/2103.15237/) (2021)

## Built on by (most cited first, 15 of 15)

- [LLMs can Find Mathematical Reasoning Mistakes by Pedagogical Chain-of-Thought](https://mlsanity.com/paper/2405.06705/) (2024)
- [Automatic Short Math Answer Grading via In-context Meta-learning](https://mlsanity.com/paper/2205.15219/) (2022)
- [Can Large Language Models Make the Grade? An Empirical Study Evaluating LLMs Ability to Mark Short Answer Questions in K-12 Education](https://mlsanity.com/paper/2405.02985/) (2024)
- [Can LLMs Grade Short-Answer Reading Comprehension Questions : An Empirical Study with a Novel Dataset](https://mlsanity.com/paper/2310.18373/) (2023)
- [Autograding Mathematical Induction Proofs with Natural Language Processing](https://mlsanity.com/paper/2406.10268/) (2024)
- [G-SciEdBERT: A Contextualized LLM for Science Assessment Tasks in German](https://mlsanity.com/paper/2402.06584/) (2024)
- [Interpreting Latent Student Knowledge Representations in Programming Assignments](https://mlsanity.com/paper/2405.08213/) (2024)
- [Exploration of Summarization by Generative Language Models for Automated Scoring of Long Essays](https://mlsanity.com/paper/2510.22830/) (2025)
- [WeatherFormer: A Pretrained Encoder Model for Learning Robust Weather Representations from Small Datasets](https://mlsanity.com/paper/2405.17455/) (2024)
- [Efficient Multi-Task Inferencing with a Shared Backbone and Lightweight Task-Specific Adapters for Automatic Scoring](https://mlsanity.com/paper/2412.21065/) (2024)
- [Modeling and Analyzing Scorer Preferences in Short-Answer Math Questions](https://mlsanity.com/paper/2306.00791/) (2023)
- [Learning to Love Edge Cases in Formative Math Assessment: Using the AMMORE Dataset and Chain-of-Thought Prompting to Improve Grading Accuracy](https://mlsanity.com/paper/2409.17904/) (2024)
- [Empirical Comparison of Encoder-Based Language Models and Feature-Based Supervised Machine Learning Approaches to Automated Scoring of Long Essays](https://mlsanity.com/paper/2601.02659/) (2026)
- [S-GRADES -- Studying Generalization of Student Response Assessments in Diverse Evaluative Settings](https://mlsanity.com/paper/2603.10233/) (2026)
- [Gumbel Machine: Counterfactual Student Writing Generation via Gumbel Noise Steering](https://mlsanity.com/paper/2605.27249/) (2026)

## Abstract (arXiv)

Automated scoring of open-ended student responses has the potential to significantly reduce human grader effort. Recent advances in automated scoring often leverage textual representations based on pre-trained language models such as BERT and GPT as input to scoring models. Most existing approaches train a separate model for each item/question, which is suitable for scenarios such as essay scoring where items can be quite different from one another. However, these approaches have two limitations: 1) they fail to leverage item linkage for scenarios such as reading comprehension where multiple items may share a reading passage; 2) they are not scalable since storing one model per item becomes difficult when models have a large number of parameters. In this paper, we report our (grand prize-winning) solution to the National Assessment of Education Progress (NAEP) automated scoring challenge for reading comprehension. Our approach, in-context BERT fine-tuning, produces a single shared scoring model for all items with a carefully-designed input structure to provide contextual information on each item. We demonstrate the effectiveness of our approach via local evaluations using the training dataset provided by the challenge. We also discuss the biases, common error types, and limitations of our approach.