# SLIDE: A Framework Integrating Small and Large Language Models for Open-Domain Dialogues Evaluation

arXiv: https://arxiv.org/abs/2405.15924 · HTML: https://mlsanity.com/paper/2405.15924/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 40 references that are in the map.
- First posted on arXiv 24 May 2024.

## Builds on (most cited first, 26 of 26)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [Towards a Unified Multi-Dimensional Evaluator for Text Generation](https://mlsanity.com/paper/2210.07197/) (2022)
- [Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations](https://mlsanity.com/paper/2308.11995/) (2023)
- [RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems](https://mlsanity.com/paper/1701.03079/) (2017)
- [Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots](https://mlsanity.com/paper/2004.03588/) (2020)
- [Towards Universal Paraphrastic Sentence Embeddings](https://mlsanity.com/paper/1511.08198/) (2015)
- [Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretraining](https://mlsanity.com/paper/2009.11321/) (2020)
- [Evaluating Evaluation Metrics: A Framework for Analyzing NLG Evaluation Metrics using Measurement Theory](https://mlsanity.com/paper/2305.14889/) (2023)
- [Effective Distillation of Table-based Reasoning Ability from LLMs](https://mlsanity.com/paper/2309.13182/) (2023)
- [Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation](https://mlsanity.com/paper/2106.05894/) (2021)
- [Enhancing Dialogue Generation via Dynamic Graph Knowledge Aggregation](https://mlsanity.com/paper/2306.16195/) (2023)
- [MDD-Eval: Self-Training on Augmented Data for Multi-Domain Dialogue Evaluation](https://mlsanity.com/paper/2112.07194/) (2021)
- [The Iron(ic) Melting Pot: Reviewing Human Evaluation in Humour, Irony and Sarcasm Generation](https://mlsanity.com/paper/2311.05552/) (2023)
- [Fast and Scalable Dialogue State Tracking with Explicit Modular Decomposition](https://mlsanity.com/paper/2004.10663/) (2020)
- [Improving Medical Dialogue Generation with Abstract Meaning Representations](https://mlsanity.com/paper/2309.10608/) (2023)
- [Improving Biomedical Abstractive Summarisation with Knowledge Aggregation from Citation Papers](https://mlsanity.com/paper/2310.15684/) (2023)
- [A Cross-Attention Augmented Model for Event-Triggered Context-Aware Story Generation](https://mlsanity.com/paper/2311.11271/) (2023)

## Built on by (most cited first, 11 of 11)

- [X-ray Made Simple: Lay Radiology Report Generation and Robust Evaluation](https://mlsanity.com/paper/2406.17911/) (2024)
- [Crafting Customisable Characters with LLMs: A Persona-Driven Role-Playing Agent Framework](https://mlsanity.com/paper/2406.17962/) (2024)
- [GRAPHMOE: Amplifying Cognitive Depth of Mixture-of-Experts Network via Introducing Self-Rethinking Mechanism](https://mlsanity.com/paper/2501.07890/) (2025)
- [Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG Evaluation Prompts](https://mlsanity.com/paper/2504.21117/) (2025)
- [The Single-Multi Evolution Loop for Self-Improving Model Collaboration Systems](https://mlsanity.com/paper/2602.05182/) (2026)
- [ContrastScore: Towards Higher Quality, Less Biased, More Efficient Evaluation Metrics with Contrastive Evaluation](https://mlsanity.com/paper/2504.02106/) (2025)
- [DRE: An Effective Dual-Refined Method for Integrating Small and Large Language Models in Open-Domain Dialogue Evaluation](https://mlsanity.com/paper/2506.04516/) (2025)
- [Why Text Prevails: Vision May Undermine Multimodal Medical Decision Making](https://mlsanity.com/paper/2512.13747/) (2025)
- [BioMNER: A Dataset for Biomedical Method Entity Recognition](https://mlsanity.com/paper/2406.20038/) (2024)
- [Natural Language Generation](https://mlsanity.com/paper/2503.16728/) (2025)
- [FigEx2: Visual-Conditioned Panel Detection and Captioning for Scientific Compound Figures](https://mlsanity.com/paper/2601.08026/) (2026)

## Abstract (arXiv)

The long-standing one-to-many problem of gold standard responses in open-domain dialogue systems presents challenges for automatic evaluation metrics. Though prior works have demonstrated some success by applying powerful Large Language Models (LLMs), existing approaches still struggle with the one-to-many problem, and exhibit subpar performance in domain-specific scenarios. We assume the commonsense reasoning biases within LLMs may hinder their performance in domainspecific evaluations. To address both issues, we propose a novel framework SLIDE (Small and Large Integrated for Dialogue Evaluation), that leverages both a small, specialised model (SLM), and LLMs for the evaluation of open domain dialogues. Our approach introduces several techniques: (1) Contrastive learning to differentiate between robust and non-robust response embeddings; (2) A novel metric for semantic sensitivity that combines embedding cosine distances with similarity learned through neural networks, and (3) a strategy for incorporating the evaluation results from both the SLM and LLMs. Our empirical results demonstrate that our approach achieves state-of-the-art performance in both the classification and evaluation tasks, and additionally the SLIDE evaluator exhibits better correlation with human judgements. Our code is available at https:// github.com/hegehongcha/SLIDE-ACL2024.