# Better Peer Grading through Bayesian Inference

arXiv: https://arxiv.org/abs/2209.01242 · HTML: https://mlsanity.com/paper/2209.01242/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 35 references that are in the map.
- First posted on arXiv 2 Sep 2022.

## Builds on (most cited first, 11 of 11)

- [Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC](https://mlsanity.com/paper/1507.04544/) (2015)
- [Optimization of Scoring Rules](https://mlsanity.com/paper/2007.02905/) (2020)
- [Informed Truthfulness in Multi-Task Peer Prediction](https://mlsanity.com/paper/1603.03151/) (2016)
- [Tuned Models of Peer Assessment in MOOCs](https://mlsanity.com/paper/1307.2579/) (2013)
- [The PeerRank Method for Peer Assessment](https://mlsanity.com/paper/1405.7192/) (2014)
- [Measurement Integrity in Peer Prediction: A Peer Assessment Case Study](https://mlsanity.com/paper/2108.05521/) (2021)
- [Deep Bayesian Trust : A Dominant and Fair Incentive Mechanism for Crowd](https://mlsanity.com/paper/1804.05560/) (2018)
- [Report-Sensitive Spot-Checking in Peer-Grading Systems](https://mlsanity.com/paper/1906.05884/) (2019)
- [Surrogate Scoring Rules](https://mlsanity.com/paper/1802.09158/) (2018)
- [Mechanisms for Making Crowds Truthful](https://mlsanity.com/paper/1401.3451/) (2014)
- [Putting Peer Prediction Under the Micro(economic)scope and Making Truth-telling Focal](https://mlsanity.com/paper/1603.07319/) (2016)

## Built on by (most cited first, 1 of 1)

- [Manipulation and Peer Mechanisms: A Survey](https://mlsanity.com/paper/2210.01984/) (2022)

## Abstract (arXiv)

Peer grading systems aggregate noisy reports from multiple students to approximate a true grade as closely as possible. Most current systems either take the mean or median of reported grades; others aim to estimate students' grading accuracy under a probabilistic model. This paper extends the state of the art in the latter approach in three key ways: (1) recognizing that students can behave strategically (e.g., reporting grades close to the class average without doing the work); (2) appropriately handling censored data that arises from discrete-valued grading rubrics; and (3) using mixed integer programming to improve the interpretability of the grades assigned to students. We show how to make Bayesian inference practical in this model and evaluate our approach on both synthetic and real-world data obtained by using our implemented system in four large classes. These extensive experiments show that grade aggregation using our model accurately estimates true grades, students' likelihood of submitting uninformative grades, and the variation in their inherent grading error; we also characterize our models' robustness.