mlsanityData to 7 Oct 2026

Vera: A General-Purpose Plausibility Estimation Model for Commonsense Statements

arXiv 2305.03695 ↗First posted 5 May 2023cs.CL cs.AIcited by 44

Not on the map. Why?

Abstract (arXiv)

Despite the much discussed capabilities of today's language models, they are still prone to silly and unexpected commonsense failures. We consider a retrospective verification approach that reflects on the correctness of LM outputs, and introduce Vera, a general-purpose model that estimates the plausibility of declarative statements based on commonsense knowledge. Trained on ~7M commonsense statements created from 19 QA datasets and two large-scale knowledge bases, and with a combination of three training objectives, Vera is a versatile model that effectively separates correct from incorrect statements across diverse commonsense domains. When applied to solving commonsense problems in the verification format, Vera substantially outperforms existing models that can be repurposed for commonsense verification, and it further exhibits generalization capabilities to unseen tasks and provides well-calibrated outputs. We find that Vera excels at filtering LM-generated commonsense knowledge and is useful in detecting erroneous commonsense statements generated by models like ChatGPT in real-world settings.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 56 of its 75 references that are in the map. First posted on arXiv 5 May 2023.

Builds on

56 of 75 references.

References · 56

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2017On Calibration of Modern Neural Networkscited by 4,860
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
38 more
  1. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  2. 2020Supervised Contrastive Learningcited by 2,803
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  4. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  5. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  6. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  7. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  8. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  9. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  10. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  11. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  12. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  13. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  14. 2019Abductive Commonsense Reasoningcited by 300
  15. 2022Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofscited by 278
  16. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  17. 2020COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphscited by 254
  18. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 248
  19. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213
  20. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  21. 2023A Categorical Archive of ChatGPT Failurescited by 167
  22. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  23. 2020Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Modelscited by 110
  24. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 108
  25. 2021UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmarkcited by 98
  26. 2019QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questionscited by 89
  27. 2021CREAK: A Dataset for Commonsense Reasoning over Entity Knowledgecited by 88
  28. 2022UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Trainingcited by 57
  29. 2022Generating Natural Language Proofs with Verifier-Guided Searchcited by 52
  30. 2020GenericsKB: A Knowledge Base of Generic Statementscited by 51
  31. 2021PROST: Physical Reasoning of Objects through Space and Timecited by 48
  32. 2018QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationshipscited by 46
  33. 2021Can NLI Models Verify QA Systems' Predictions?cited by 46
  34. 2022Rainier: Reinforced Knowledge Introspector for Commonsense Question Answeringcited by 43
  35. 2020SemEval-2020 Task 4: Commonsense Validation and Explanationcited by 42
  36. 2022Things not Written in Text: Exploring Spatial Commonsense from Visual Signalscited by 41
  37. 2021COM2SENSE: A Commonsense Reasoning Benchmark with Complementary Sentencescited by 40
  38. 2021General-Purpose Question-Answering with Macawcited by 40

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 22; 2025: 13; 2026: 1.

44 papers.

Sort:

By area:

  1. 2023SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionalitycited by 238
  2. 2025Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Abovecited by 51
  3. 2024Learn "No" to Say "Yes" Better: Improving Vision-Language Models via Negationscited by 45
  4. 2024V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuningcited by 45
  5. 2023Contrasting Intra-Modal and Ranking Cross-Modal Hard Negatives to Enhance Visio-Linguistic Compositional Understandingcited by 40
  6. 2024Establishing Task Scaling Laws via Compute-Efficient Model Ladderscited by 32
  7. 2024Logically Consistent Language Models via Neuro-Symbolic Integrationcited by 30
  8. 2023Tailoring Self-Rationalizers with Multi-Reward Distillationcited by 21
  9. 2024FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehensioncited by 20
  10. 2024An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaborationcited by 16
  11. 2024FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correctioncited by 16
  12. 2024NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Modelscited by 13
32 more
  1. 2024MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Datasetcited by 11
  2. 2024IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commercecited by 11
  3. 2024MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understandingcited by 10
  4. 2024VELOCITI: Benchmarking Video-Language Compositional Reasoning with Strict Entailmentcited by 10
  5. 2025EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Associationcited by 10
  6. 2024Varying Shades of Wrong: Aligning LLMs with Wrong Answers Onlycited by 9
  7. 2025Decoupled Global-Local Alignment for Improving Compositional Understandingcited by 9
  8. 2023NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillationcited by 8
  9. 2024ComparisonQA: Evaluating Factuality Robustness of LLMs Through Knowledge Frequency Control and Uncertaintycited by 8
  10. 2025MuSLR: Multimodal Symbolic Logical Reasoningcited by 7
  11. 2023An Examination of the Compositionality of Large Generative Vision-Language Modelscited by 6
  12. 2024Information-Theoretic Distillation for Reference-less Summarizationcited by 6
  13. 2024BiVLC: Extending Vision-Language Compositionality Evaluation with Text-to-Image Retrievalcited by 6
  14. 2024Intention Knowledge Graph Construction for User Intention Relation Modelingcited by 6
  15. 2025A Good CREPE needs more than just Sugar: Investigating Biases in Compositional Vision-Language Benchmarkscited by 5
  16. 2024Knowledge Editing on Black-box Large Language Modelscited by 4
  17. 2024On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directionscited by 3
  18. 2023Knowledge Rumination for Pre-trained Language Modelscited by 2
  19. 2023Are LLMs Rigorous Logical Reasoners? Empowering Natural Language Proof Generation by Stepwise Decoding with Contrastive Learningcited by 2
  20. 2024EcomEdit: An Automated E-commerce Knowledge Editing Framework for Enhanced Product and Purchase Intention Understandingcited by 2
  21. 2025Is a Peeled Apple Still Red? Evaluating LLMs' Ability for Conceptual Combination with Property Typecited by 2
  22. 2025SCRAMBLe : Enhancing Multimodal LLM Compositionality with Synthetic Preference Datacited by 2
  23. 2025Visual Perturbation and Adaptive Hard Negative Contrastive Learning for Compositional Reasoning in Vision-Language Modelscited by 2
  24. 2025Structuring the Unstructured: A Systematic Review of Text-to-Structure Generation for Agentic AI with a Universal Evaluation Frameworkcited by 2
  25. 2023BOOST: Harnessing Black-Box Control to Boost Commonsense in LMs' Generationcited by 1
  26. 2024Rule or Story, Which is a Better Commonsense Expression for Talking with Large Language Models?cited by 1
  27. 2025Estimating Commonsense Plausibility through Semantic Shiftscited by 1
  28. 2025Arguments that Alter Minds: LLM Rationales Sway Human (and LLM) Notions of Plausibilitycited by 1
  29. 2024Counterfactual Samples Constructing and Training for Commonsense Statements Estimationcited by 0
  30. 2025Modelling Adjectival Modification Effects on Semantic Plausibilitycited by 0
  31. 2025Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanationscited by 0
  32. 2026The Maskability Index: Predicting Task-Objective Alignment in Pretrained Language Modelscited by 0