# An Interpretable and Crosslingual Method for Evaluating Second-Language Dialogues

arXiv: https://arxiv.org/abs/2408.16518 · HTML: https://mlsanity.com/paper/2408.16518/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 39 references that are in the map.
- First posted on arXiv 29 Aug 2024.

## Builds on (most cited first, 4 of 4)

- [GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems](https://mlsanity.com/paper/2010.03994/) (2020)
- [Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents](https://mlsanity.com/paper/2201.04723/) (2022)
- [FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows](https://mlsanity.com/paper/2202.06633/) (2022)
- [Interaction Matters: An Evaluation Framework for Interactive Dialogue Assessment on English Second Language Conversations](https://mlsanity.com/paper/2407.06479/) (2024)

## Built on by (most cited first, 2 of 2)

- ['No' Matters: Out-of-Distribution Detection in Multimodality Long Dialogue](https://mlsanity.com/paper/2410.23883/) (2024)
- [Can LLMs Simulate L2-English Dialogue? An Information-Theoretic Analysis of L1-Dependent Biases](https://mlsanity.com/paper/2502.14507/) (2025)

## Abstract (arXiv)

We analyse the cross-lingual transferability of a dialogue evaluation framework that assesses the relationships between micro-level linguistic features (e.g. backchannels) and macro-level interactivity labels (e.g. topic management), originally designed for English-as-a-second-language dialogues. To this end, we develop CNIMA (Chinese Non-Native Interactivity Measurement and Automation), a Chinese-as-a-second-language labelled dataset with 10K dialogues. We found the evaluation framework to be robust across distinct languages: English and Chinese, revealing language-specific and language-universal relationships between micro-level and macro-level features. Next, we propose an automated, interpretable approach with low data requirement that scores the overall quality of a second-language dialogue based on the framework. Our approach is interpretable in that it reveals the key linguistic and interactivity features that contributed to the overall quality score. As our approach does not require labelled data, it can also be adapted to other languages for second-language dialogue evaluation.