# Combining physics education and machine learning research to measure evidence of students' mechanistic sensemaking

arXiv: https://arxiv.org/abs/2503.15638 · HTML: https://mlsanity.com/paper/2503.15638/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 60 references that are in the map.
- First posted on arXiv 19 Mar 2025.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [Towards Understanding and Mitigating Social Biases in Language Models](https://mlsanity.com/paper/2106.13219/) (2021)
- [Explicit Inductive Bias for Transfer Learning with Convolutional Networks](https://mlsanity.com/paper/1802.01483/) (2018)
- [Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning](https://mlsanity.com/paper/1811.12808/) (2018)
- [Revisiting Small Batch Training for Deep Neural Networks](https://mlsanity.com/paper/1804.07612/) (2018)
- [Evaluating Large Language Models in Analysing Classroom Dialogue](https://mlsanity.com/paper/2402.02380/) (2024)
- [Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning](https://mlsanity.com/paper/2404.07546/) (2024)
- [An Attention Matrix for Every Decision: Faithfulness-based Arbitration Among Multiple Attention-Based Interpretations of Transformers in Text Classification](https://mlsanity.com/paper/2209.10876/) (2022)

## Built on by (most cited first, 1 of 1)

- [Locating acts of mechanistic reasoning in student team conversations with mechanistic machine learning](https://mlsanity.com/paper/2604.21870/) (2026)

## Abstract (arXiv)

Advances in machine learning (ML) offer new possibilities for science education research. We report on early progress in the design of an ML-based tool to analyze students' mechanistic sensemaking, working from a coding scheme that is aligned with previous work in physics education research (PER) and amenable to recently developed ML classification strategies using language encoders. We describe pilot tests of the tool, in three versions with different language encoders, to analyze sensemaking evident in college students' written responses to brief conceptual questions. The results show, first, that the tool's measurements of sensemaking can achieve useful agreement with a human coder, and, second, that encoder design choices entail a tradeoff between accuracy and computational expense. We discuss the promise and limitations of this approach, providing examples as to how this measurement scheme may serve PER in the future. We conclude with reflections on the use of ML to support PER research, with cautious optimism for strategies of co-design between PER and ML.