mlsanityData to 14 Oct 2026

ACCENT: An Automatic Event Commonsense Evaluation Metric for Open-Domain Dialogue Systems

arXiv 2305.07797 ↗First posted 12 May 2023cs.CLcited by 11

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Commonsense reasoning is omnipresent in human communications and thus is an important feature for open-domain dialogue systems. However, evaluating commonsense in dialogue systems is still an open challenge. We take the first step by focusing on event commonsense that considers events and their relations, and is crucial in both dialogues and general commonsense reasoning. We propose ACCENT, an event commonsense evaluation metric empowered by commonsense knowledge bases (CSKBs). ACCENT first extracts event-relation tuples from a dialogue, and then evaluates the response by scoring the tuples in terms of their compatibility with the CSKB. To evaluate ACCENT, we construct the first public event commonsense evaluation dataset for open-domain dialogues. Our experiments show that ACCENT is an efficient metric for event commonsense evaluation, which achieves higher correlations with human judgments than existing baselines.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 35 of its 65 references that are in the map. First posted on arXiv 12 May 2023.

Builds on

35 of 65 references.

References · 35

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  6. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  7. 2019Language Models as Knowledge Bases?cited by 1,936
  8. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  9. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  10. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
  11. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 652
  12. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
23 more
  1. 2019KG-BERT: BERT for Knowledge Graph Completioncited by 324
  2. 2020COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphscited by 254
  3. 2017A Survey on Dialogue Systems: Recent Advances and New Frontierscited by 162
  4. 2019Challenges in Building Intelligent Open-domain Dialog Systemscited by 147
  5. 2020Unsupervised Evaluation of Interactive Dialog with DialoGPTcited by 139
  6. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 132
  7. 2019Unifying Human and Statistical Evaluation for Natural Language Generationcited by 127
  8. 2019ASER: A Large-scale Eventuality Knowledge Graphcited by 89
  9. 2017RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systemscited by 88
  10. 2020GLUCOSE: GeneraLized and COntextualized Story Explanationscited by 86
  11. 2018Event2Mind: Commonsense Inference on Events, Intents, and Reactionscited by 82
  12. 2020PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learningcited by 79
  13. 2021ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embeddingcited by 74
  14. 2020TransOMCS: From Linguistic Graphs to Commonsense Knowledgecited by 52
  15. 2019Predictive Engagement: An Efficient Metric For Automatic Evaluation of Open-Domain Dialogue Systemscited by 36
  16. 2020Towards Unified Dialogue System Evaluation: A Comprehensive Analysis of Current Evaluation Protocolscited by 35
  17. 2021Commonsense-Focused Dialogues for Response Generation: An Empirical Studycited by 30
  18. 2018Augmenting Neural Response Generation with Context-Aware Topical Attentioncited by 27
  19. 2021DISCOS: Bridging the Gap between Discourse Knowledge and Commonsense Knowledgecited by 26
  20. 2022DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulationscited by 25
  21. 2022What is wrong with you?: Leveraging User Sentiment for Automatic Dialog Evaluationcited by 18
  22. 2021Benchmarking Commonsense Knowledge Base Population with an Effective Evaluation Datasetcited by 15
  23. 2021DiSCoL: Toward Engaging Dialogue Systems through Conversational Line Guided Response Generationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 1; 2025: 5; 2026: 1.

11 papers.

  1. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
  2. 2023Leveraging Large Language Models for Automated Dialogue Analysiscited by 14
  3. 2023Class-Incremental Learning based on Label Generationcited by 11
  4. 2025Robust Misinformation Detection by Visiting Potential Commonsense Conflictcited by 3
  5. 2025AgentCrypt: Advancing Privacy and (Secure) Computation in AI Agent Collaborationcited by 2
  6. 2023Open-Domain Text Evaluation via Contrastive Distribution Methodscited by 1
  7. 2023BOOST: Harnessing Black-Box Control to Boost Commonsense in LMs' Generationcited by 1
  8. 2025Estimating Commonsense Plausibility through Semantic Shiftscited by 1
  9. 2025Reasoning in Action: MCTS-Driven Knowledge Retrieval for Large Language Modelscited by 1
  10. 2025Commonsense Generation and Evaluation for Dialogue Systems using Large Language Modelscited by 0
  11. 2026The Maskability Index: Predicting Task-Objective Alignment in Pretrained Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Leveraging Explicit Reasoning for Inference Integration in Commonsense-Augmented Dialogue Models
  2. 2023CORECODE: A Common Sense Annotated Dialogue Dataset with Benchmark Tasks for Chinese Large Language Models
  3. 2023SYNDICOM: Improving Conversational Commonsense with Error-Injection and Natural Language Feedback
  4. 2023Commonsense Reasoning for Conversational AI: A Survey of the State of the Art
  5. 2024ConvoSense: Overcoming Monotonous Commonsense Inferences for Conversational AI
  6. 2024SLIDE: A Framework Integrating Small and Large Language Models for Open-Domain Dialogues Evaluation
  7. 2025MEDAL: A Framework for Benchmarking LLMs as Multilingual Open-Domain Dialogue Evaluators
  8. 2026SKG-Eval: Stateful Evaluation of Multi-Turn Dialogue via Incremental Semantic Knowledge Graphs