mlsanityData to 14 Oct 2026

Zero-Shot Commonsense Validation and Reasoning with Large Language Models: An Evaluation on SemEval-2020 Task 4 Dataset

arXiv 2502.15810 ↗First posted 19 Feb 2025cs.CL cs.AIcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

This study evaluates the performance of Large Language Models (LLMs) on SemEval-2020 Task 4 dataset, focusing on commonsense validation and explanation. Our methodology involves evaluating multiple LLMs, including LLaMA3-70B, Gemma2-9B, and Mixtral-8x7B, using zero-shot prompting techniques. The models are tested on two tasks: Task A (Commonsense Validation), where models determine whether a statement aligns with commonsense knowledge, and Task B (Commonsense Explanation), where models identify the reasoning behind implausible statements. Performance is assessed based on accuracy, and results are compared to fine-tuned transformer-based models. The results indicate that larger models outperform previous models and perform closely to human evaluation for Task A, with LLaMA3-70B achieving the highest accuracy of 98.40% in Task A whereas, lagging behind previous models with 93.40% in Task B. However, while models effectively identify implausible statements, they face challenges in selecting the most relevant explanation, highlighting limitations in causal and inferential reasoning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 13 of its 21 references that are in the map. First posted on arXiv 19 Feb 2025.

Builds on

13 of 21 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey2 Apr

Other references · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  4. 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 304
  5. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  6. 2020A Survey on Causal Inferencecited by 202
  7. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 109
  8. 2021A Systematic Investigation of Commonsense Knowledge in Large Language Modelscited by 56
  9. 2020SemEval-2020 Task 4: Commonsense Validation and Explanationcited by 42
  10. 2024Right for Right Reasons: Large Language Models for Verifiable Commonsense Knowledge Graph Question Answeringcited by 11
  11. 2020IIE-NLP-NUT at SemEval-2020 Task 4: Guiding PLM with Prompt Template Reconstruction Strategy for ComVEcited by 4
  12. 2020ECNU-SenseMaker at SemEval-2020 Task 4: Leveraging Heterogeneous Knowledge Resources for Commonsense Validation and Explanationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Evaluate Confidence Instead of Perplexity for Zero-shot Commonsense Reasoning
  2. 2024From Data to Commonsense Reasoning: The Use of Large Language Models for Explainable AI
  3. 2025Reasoning Capabilities and Invariability of Large Language Models
  4. 2025HellaSwag-Pro: A Large-Scale Bilingual Benchmark for Evaluating the Robustness of LLMs in Commonsense Reasoning
  5. 2025Who Benchmarks the Benchmarks? Towards Comprehensive Evaluation of Commonsense Reasoning Benchmarks
  6. 2023Large Language Models Are Also Good Prototypical Commonsense Reasoners
  7. 2022Large Language Models are Zero-Shot Reasoners
  8. 2024LINKED: Eliciting, Filtering and Integrating Knowledge in Large Language Model for Commonsense Reasoning