# Interaction Matters: An Evaluation Framework for Interactive Dialogue Assessment on English Second Language Conversations

arXiv: https://arxiv.org/abs/2407.06479 · HTML: https://mlsanity.com/paper/2407.06479/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 31 references that are in the map.
- First posted on arXiv 9 Jul 2024.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [The Second Conversational Intelligence Challenge (ConvAI2)](https://mlsanity.com/paper/1902.00098/) (2019)
- [RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems](https://mlsanity.com/paper/1701.03079/) (2017)
- [Learning an Unreferenced Metric for Online Dialogue Evaluation](https://mlsanity.com/paper/2005.00583/) (2020)
- [DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations](https://mlsanity.com/paper/2203.09711/) (2022)
- [Interactive Evaluation of Dialog Track at DSTC9](https://mlsanity.com/paper/2207.14403/) (2022)
- [Clozer: Adaptable Data Augmentation for Cloze-style Reading Comprehension](https://mlsanity.com/paper/2203.16027/) (2022)

## Built on by (most cited first, 4 of 4)

- [WHoW: A Cross-domain Approach for Analysing Conversation Moderation](https://mlsanity.com/paper/2410.15551/) (2024)
- [An Interpretable and Crosslingual Method for Evaluating Second-Language Dialogues](https://mlsanity.com/paper/2408.16518/) (2024)
- ['No' Matters: Out-of-Distribution Detection in Multimodality Long Dialogue](https://mlsanity.com/paper/2410.23883/) (2024)
- [Can LLMs Simulate L2-English Dialogue? An Information-Theoretic Analysis of L1-Dependent Biases](https://mlsanity.com/paper/2502.14507/) (2025)

## Abstract (arXiv)

We present an evaluation framework for interactive dialogue assessment in the context of English as a Second Language (ESL) speakers. Our framework collects dialogue-level interactivity labels (e.g., topic management; 4 labels in total) and micro-level span features (e.g., backchannels; 17 features in total). Given our annotated data, we study how the micro-level features influence the (higher level) interactivity quality of ESL dialogues by constructing various machine learning-based models. Our results demonstrate that certain micro-level features strongly correlate with interactivity quality, like reference word (e.g., she, her, he), revealing new insights about the interaction between higher-level dialogue quality and lower-level linguistic signals. Our framework also provides a means to assess ESL communication, which is useful for language assessment.