mlsanityData to 7 Oct 2026

Autoencoding Language Model Based Ensemble Learning for Commonsense Validation and Explanation

arXiv 2204.03324 ↗First posted 7 Apr 2022cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

An ultimate goal of artificial intelligence is to build computer systems that can understand human languages. Understanding commonsense knowledge about the world expressed in text is one of the foundational and challenging problems to create such intelligent systems. As a step towards this goal, we present in this paper ALMEn, an Autoencoding Language Model based Ensemble learning method for commonsense validation and explanation. By ensembling several advanced pre-trained language models including RoBERTa, DeBERTa, and ELECTRA with Siamese neural networks, our method can distinguish natural language statements that are against commonsense (validation subtask) and correctly identify the reason for making against commonsense (explanation selection subtask). Experimental results on the benchmark dataset of SemEval-2020 Task 4 show that our method outperforms state-of-the-art models, reaching 97.9% and 95.4% accuracies on the validation and explanation selection subtasks, respectively.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 7 Apr 2022.

Builds on

23 of 33 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  7. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  8. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  9. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  11. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  12. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
11 more
  1. 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 303
  2. 2020MuTual: A Dataset for Multi-Turn Dialogue Reasoningcited by 114
  3. 2020SemEval-2020 Task 4: Commonsense Validation and Explanationcited by 42
  4. 2016Ordinal Common-sense Inferencecited by 35
  5. 2018Modeling Semantic Plausibility by Injecting World Knowledgecited by 24
  6. 2020IIE-NLP-NUT at SemEval-2020 Task 4: Guiding PLM with Prompt Template Reconstruction Strategy for ComVEcited by 4
  7. 2020ECNU-SenseMaker at SemEval-2020 Task 4: Leveraging Heterogeneous Knowledge Resources for Commonsense Validation and Explanationcited by 4
  8. 2020CS-NLP team at SemEval-2020 Task 4: Evaluation of State-of-the-art NLP Deep Learning Architectures on Commonsense Reasoning Taskcited by 3
  9. 2020QiaoNing at SemEval-2020 Task 4: Commonsense Validation and Explanation system based on ensemble of language modelcited by 3
  10. 2020LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation using Pretraining Language Modelcited by 1
  11. 2020CS-NET at SemEval-2020 Task 4: Siamese BERT for ComVEcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2023Speculative Decoding with Big Little Decodercited by 163
  2. 2024Cost-Effective Online Multi-LLM Selection with Versatile Reward Modelscited by 36