# ACCENT: An Automatic Event Commonsense Evaluation Metric for Open-Domain Dialogue Systems

arXiv: https://arxiv.org/abs/2305.07797 · HTML: https://mlsanity.com/paper/2305.07797/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 35 of its 65 references that are in the map.
- First posted on arXiv 12 May 2023.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 30 of 35)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://mlsanity.com/paper/1906.05317/) (2019)
- [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019)
- [COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs](https://mlsanity.com/paper/2010.05953/) (2020)
- [A Survey on Dialogue Systems: Recent Advances and New Frontiers](https://mlsanity.com/paper/1711.01731/) (2017)
- [Challenges in Building Intelligent Open-domain Dialog Systems](https://mlsanity.com/paper/1905.05709/) (2019)
- [Unsupervised Evaluation of Interactive Dialog with DialoGPT](https://mlsanity.com/paper/2006.12719/) (2020)
- [DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings](https://mlsanity.com/paper/2204.10298/) (2022)
- [Unifying Human and Statistical Evaluation for Natural Language Generation](https://mlsanity.com/paper/1904.02792/) (2019)
- [ASER: A Large-scale Eventuality Knowledge Graph](https://mlsanity.com/paper/1905.00270/) (2019)
- [RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems](https://mlsanity.com/paper/1701.03079/) (2017)
- [GLUCOSE: GeneraLized and COntextualized Story Explanations](https://mlsanity.com/paper/2009.07758/) (2020)
- [Event2Mind: Commonsense Inference on Events, Intents, and Reactions](https://mlsanity.com/paper/1805.06939/) (2018)
- [PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning](https://mlsanity.com/paper/2006.16779/) (2020)
- [ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding](https://mlsanity.com/paper/2109.04380/) (2021)
- [TransOMCS: From Linguistic Graphs to Commonsense Knowledge](https://mlsanity.com/paper/2005.00206/) (2020)
- [Predictive Engagement: An Efficient Metric For Automatic Evaluation of Open-Domain Dialogue Systems](https://mlsanity.com/paper/1911.01456/) (2019)
- [Towards Unified Dialogue System Evaluation: A Comprehensive Analysis of Current Evaluation Protocols](https://mlsanity.com/paper/2006.06110/) (2020)
- [Commonsense-Focused Dialogues for Response Generation: An Empirical Study](https://mlsanity.com/paper/2109.06427/) (2021)
- [Augmenting Neural Response Generation with Context-Aware Topical Attention](https://mlsanity.com/paper/1811.01063/) (2018)

## Built on by (most cited first, 11 of 11)

- [PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action](https://mlsanity.com/paper/2409.00138/) (2024)
- [Leveraging Large Language Models for Automated Dialogue Analysis](https://mlsanity.com/paper/2309.06490/) (2023)
- [Class-Incremental Learning based on Label Generation](https://mlsanity.com/paper/2306.12619/) (2023)
- [Robust Misinformation Detection by Visiting Potential Commonsense Conflict](https://mlsanity.com/paper/2504.21604/) (2025)
- [AgentCrypt: Advancing Privacy and (Secure) Computation in AI Agent Collaboration](https://mlsanity.com/paper/2512.08104/) (2025)
- [Open-Domain Text Evaluation via Contrastive Distribution Methods](https://mlsanity.com/paper/2306.11879/) (2023)
- [BOOST: Harnessing Black-Box Control to Boost Commonsense in LMs' Generation](https://mlsanity.com/paper/2310.17054/) (2023)
- [Estimating Commonsense Plausibility through Semantic Shifts](https://mlsanity.com/paper/2502.13464/) (2025)
- [Reasoning in Action: MCTS-Driven Knowledge Retrieval for Large Language Models](https://mlsanity.com/paper/2601.00003/) (2025)
- [Commonsense Generation and Evaluation for Dialogue Systems using Large Language Models](https://mlsanity.com/paper/2506.19483/) (2025)
- [The Maskability Index: Predicting Task-Objective Alignment in Pretrained Language Models](https://mlsanity.com/paper/2607.20265/) (2026)

## Abstract (arXiv)

Commonsense reasoning is omnipresent in human communications and thus is an important feature for open-domain dialogue systems. However, evaluating commonsense in dialogue systems is still an open challenge. We take the first step by focusing on event commonsense that considers events and their relations, and is crucial in both dialogues and general commonsense reasoning. We propose ACCENT, an event commonsense evaluation metric empowered by commonsense knowledge bases (CSKBs). ACCENT first extracts event-relation tuples from a dialogue, and then evaluates the response by scoring the tuples in terms of their compatibility with the CSKB. To evaluate ACCENT, we construct the first public event commonsense evaluation dataset for open-domain dialogues. Our experiments show that ACCENT is an efficient metric for event commonsense evaluation, which achieves higher correlations with human judgments than existing baselines.