# Modelling Adjectival Modification Effects on Semantic Plausibility

arXiv: https://arxiv.org/abs/2507.21828 · HTML: https://mlsanity.com/paper/2507.21828/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 43 references that are in the map.
- First posted on arXiv 29 Jul 2025.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks](https://mlsanity.com/paper/2010.08240/) (2020)
- [Event knowledge in large language models: the gap between the impossible and the unlikely](https://mlsanity.com/paper/2212.01488/) (2022)
- [Vera: A General-Purpose Plausibility Estimation Model for Commonsense Statements](https://mlsanity.com/paper/2305.03695/) (2023)
- [SemEval-2020 Task 4: Commonsense Validation and Explanation](https://mlsanity.com/paper/2007.00236/) (2020)
- [deep-significance - Easy and Meaningful Statistical Significance Testing in the Age of Neural Networks](https://mlsanity.com/paper/2204.06815/) (2022)
- [Modeling Semantic Plausibility by Injecting World Knowledge](https://mlsanity.com/paper/1804.00619/) (2018)
- [SMedBERT: A Knowledge-Enhanced Pre-trained Language Model with Structured Semantics for Medical Text Mining](https://mlsanity.com/paper/2108.08983/) (2021)
- [SemEval-2024 Shared Task 6: SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes](https://mlsanity.com/paper/2403.07726/) (2024)
- [Modeling Event Plausibility with Consistent Conceptual Abstraction](https://mlsanity.com/paper/2104.10247/) (2021)
- [Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge](https://mlsanity.com/paper/2107.10922/) (2021)
- [The Possible, the Plausible, and the Desirable: Event-Based Modality Detection for Language Processing](https://mlsanity.com/paper/2106.08037/) (2021)
- [Learning to Rank for Plausible Plausibility](https://mlsanity.com/paper/1906.02079/) (2019)
- [Can a Gorilla Ride a Camel? Learning Semantic Plausibility from Text](https://mlsanity.com/paper/1911.05689/) (2019)
- [Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses](https://mlsanity.com/paper/2305.19339/) (2023)
- [Made of Steel? Learning Plausible Materials for Components in the Vehicle Repair Domain](https://mlsanity.com/paper/2304.14745/) (2023)

## Abstract (arXiv)

While the task of assessing the plausibility of events such as "news is relevant" has been addressed by a growing body of work, less attention has been paid to capturing changes in plausibility as triggered by event modification. Understanding changes in plausibility is relevant for tasks such as dialogue generation, commonsense reasoning, and hallucination detection, as it allows to correctly model, for example, "false news is relevant", which is of lower relevance but higher concern due to potential disinformation. In this work, we tackle the Adept challenge benchmark (Emami et al. 2021) consisting of 16K English sentence pairs differing by exactly one adjectival modifier (e.g., false.) Our modeling experiments provide a conceptually novel method using sentence transformers and reveal that sentence transformers struggle despite their conceptual alignment with the task at hand, underperforming in comparison to transformers like RoBERTa. Finally, we discuss our findings in relation to prior work and present a detailed error analysis to shed light on potential sources for ST underperformance, highlighting advantages and shortcomings of the examined methods for balancing out train and test data.