# The BEA 2023 Shared Task on Generating AI Teacher Responses in Educational Dialogues

arXiv: https://arxiv.org/abs/2306.06941 · HTML: https://mlsanity.com/paper/2306.06941/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 12 Jun 2023.

## Builds on (most cited first, 16 of 16)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization](https://mlsanity.com/paper/2210.01241/) (2022)
- [A Comprehensive Assessment of Dialog Evaluation Metrics](https://mlsanity.com/paper/2106.03706/) (2021)
- [ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons](https://mlsanity.com/paper/1909.03087/) (2019)
- [The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialogues](https://mlsanity.com/paper/2205.07540/) (2022)
- [Dialogue Response Ranking Training with Large-Scale Human Feedback Data](https://mlsanity.com/paper/2009.06978/) (2020)
- [The Teacher-Student Chatroom Corpus](https://mlsanity.com/paper/2011.07109/) (2020)
- [The ADAIO System at the BEA-2023 Shared Task on Generating AI Teacher Responses in Educational Dialogues](https://mlsanity.com/paper/2306.05360/) (2023)

## Built on by (most cited first, 27 of 27)

- [EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education](https://mlsanity.com/paper/2308.02773/) (2023)
- [Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutors](https://mlsanity.com/paper/2412.09416/) (2024)
- [MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors](https://mlsanity.com/paper/2502.18940/) (2025)
- [Findings of the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors](https://mlsanity.com/paper/2507.10579/) (2025)
- [Assessing the efficacy of large language models in generating accurate teacher responses](https://mlsanity.com/paper/2307.04274/) (2023)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Pedagogy-driven Evaluation of Generative AI-powered Intelligent Tutoring Systems](https://mlsanity.com/paper/2510.22581/) (2025)
- [Exploring LLMs for Predicting Tutor Strategy and Student Outcomes in Dialogues](https://mlsanity.com/paper/2507.06910/) (2025)
- [Rewarding How Models Think Pedagogically: Integrating Pedagogical Reasoning and Thinking Rewards for LLMs in Education](https://mlsanity.com/paper/2601.14560/) (2026)
- [Identifying Fairness Issues in Automatically Generated Testing Content](https://mlsanity.com/paper/2404.15104/) (2024)
- [RETUYT-INCO at BEA 2025 Shared Task: How Far Can Lightweight Models Go in AI-powered Tutor Evaluation?](https://mlsanity.com/paper/2506.11243/) (2025)
- [Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact](https://mlsanity.com/paper/2603.00883/) (2026)
- [RAM2C: A Liberal Arts Educational Chatbot based on Retrieval-augmented Multi-role Multi-expert Collaboration](https://mlsanity.com/paper/2409.15461/) (2024)
- [BD at BEA 2025 Shared Task: MPNet Ensembles for Pedagogical Mistake Identification and Localization in AI Tutor Responses](https://mlsanity.com/paper/2506.01817/) (2025)
- [LangLingual: A Personalised, Exercise-oriented English Language Learning Tool Leveraging Large Language Models](https://mlsanity.com/paper/2510.23011/) (2025)
- [AITutor-EvalKit: Exploring the Capabilities of AI Tutors](https://mlsanity.com/paper/2512.03688/) (2025)
- [Your Students Don't Use LLMs Like You Wish They Did](https://mlsanity.com/paper/2604.23486/) (2026)
- [Findings of the First Teaching Monster Challenge: A Benchmark of Pedagogical Content Knowledge in AI Agents](https://mlsanity.com/paper/2608.08852/) (2026)
- [CodingTeachLLM: Empowering LLM's Coding Ability via AST Prior Knowledge](https://mlsanity.com/paper/2403.15426/) (2024)
- [CIKMar: A Dual-Encoder Approach to Prompt-Based Reranking in Educational Dialogue Systems](https://mlsanity.com/paper/2408.08805/) (2024)
- [Measuring the Robustness of Reference-Free Dialogue Evaluation Systems](https://mlsanity.com/paper/2501.06728/) (2025)
- [TeachBench: A Syllabus-Grounded Framework for Evaluating Teaching Ability in Large Language Models](https://mlsanity.com/paper/2601.21375/) (2026)
- [Latency and Cost of Multi-Agent Intelligent Tutoring at Scale](https://mlsanity.com/paper/2604.24110/) (2026)
- [ITAS: A Multi-Agent Architecture for LLM-Based Intelligent Tutoring](https://mlsanity.com/paper/2604.24808/) (2026)
- [Knowledge Distillation for Automated AI Tutor Evaluation](https://mlsanity.com/paper/2607.10647/) (2026)
- [IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems](https://mlsanity.com/paper/2607.23322/) (2026)
- [EduFair-Bench: Evaluating Pedagogical Fairness of LLM Tutors Across Student Demographics](https://mlsanity.com/paper/2609.12949/) (2026)

## Abstract (arXiv)

This paper describes the results of the first shared task on the generation of teacher responses in educational dialogues. The goal of the task was to benchmark the ability of generative language models to act as AI teachers, replying to a student in a teacher-student dialogue. Eight teams participated in the competition hosted on CodaLab. They experimented with a wide variety of state-of-the-art models, including Alpaca, Bloom, DialoGPT, DistilGPT-2, Flan-T5, GPT-2, GPT-3, GPT- 4, LLaMA, OPT-2.7B, and T5-base. Their submissions were automatically scored using BERTScore and DialogRPT metrics, and the top three among them were further manually evaluated in terms of pedagogical ability based on Tack and Piech (2022). The NAISTeacher system, which ranked first in both automated and human evaluation, generated responses with GPT-3.5 using an ensemble of prompts and a DialogRPT-based ranking of responses for given dialogue contexts. Despite the promising achievements of the participating teams, the results also highlight the need for evaluation metrics better suited to educational contexts.