# EduCoder: An Open-Source Annotation System for Education Transcript Data

arXiv: https://arxiv.org/abs/2507.05385 · HTML: https://mlsanity.com/paper/2507.05385/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 46 references that are in the map.
- First posted on arXiv 7 Jul 2025.

## Builds on (most cited first, 16 of 16)

- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [Reframing Human-AI Collaboration for Generating Free-Text Explanations](https://mlsanity.com/paper/2112.08674/) (2021)
- [The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs](https://mlsanity.com/paper/2501.10970/) (2025)
- [POTATO: The Portable Text Annotation Tool](https://mlsanity.com/paper/2212.08620/) (2022)
- [Contrasting Linguistic Patterns in Human and LLM-Generated News Text](https://mlsanity.com/paper/2308.09067/) (2023)
- [Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakes](https://mlsanity.com/paper/2310.10648/) (2023)
- [Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions](https://mlsanity.com/paper/2106.03873/) (2021)
- [The NCTE Transcripts: A Dataset of Elementary Math Classroom Transcripts](https://mlsanity.com/paper/2211.11772/) (2022)
- [The TalkMoves Dataset: K-12 Mathematics Lesson Transcripts Annotated for Teacher and Student Discursive Moves](https://mlsanity.com/paper/2204.09652/) (2022)
- [Evaluating Large Language Models in Analysing Classroom Dialogue](https://mlsanity.com/paper/2402.02380/) (2024)
- [LightTag: Text Annotation Platform](https://mlsanity.com/paper/2109.02320/) (2021)
- [Computationally Identifying Funneling and Focusing Questions in Classroom Discourse](https://mlsanity.com/paper/2208.04715/) (2022)
- [Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models](https://mlsanity.com/paper/2102.00036/) (2021)
- [Do large language models and humans have similar behaviors in causal inference with script knowledge?](https://mlsanity.com/paper/2311.07311/) (2023)
- [EASE: An Easily-Customized Annotation System Powered by Efficiency Enhancement Mechanisms](https://mlsanity.com/paper/2305.14169/) (2023)
- [Evaluating Large Language Models Against Human Annotators in Latent Content Analysis: Sentiment, Political Leaning, Emotional Intensity, and Sarcasm](https://mlsanity.com/paper/2501.02532/) (2025)

## Abstract (arXiv)

We introduce EduCoder, a domain-specialized tool designed to support utterance-level annotation of educational dialogue. While general-purpose text annotation tools for NLP and qualitative research abound, few address the complexities of coding education dialogue transcripts -- with diverse teacher-student and peer interactions. Common challenges include defining codebooks for complex pedagogical features, supporting both open-ended and categorical coding, and contextualizing utterances with external features, such as the lesson's purpose and the pedagogical value of the instruction. EduCoder is designed to address these challenges by providing a platform for researchers and domain experts to collaboratively define complex codebooks based on observed data. It incorporates both categorical and open-ended annotation types along with contextual materials. Additionally, it offers a side-by-side comparison of multiple annotators' responses, allowing comparison and calibration of annotations with others to improve data reliability. The system is open-source, with a demo video available.