# ImpScore: A Learnable Metric For Quantifying The Implicitness Level of Sentence

arXiv: https://arxiv.org/abs/2411.05172 · HTML: https://mlsanity.com/paper/2411.05172/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 7 Nov 2024.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection](https://mlsanity.com/paper/2203.09509/) (2022)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection](https://mlsanity.com/paper/2012.15761/) (2020)
- [Latent Hatred: A Benchmark for Understanding Implicit Hate Speech](https://mlsanity.com/paper/2109.05322/) (2021)
- [Foundational Challenges in Assuring Alignment and Safety of Large Language Models](https://mlsanity.com/paper/2404.09932/) (2024)
- [Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition](https://mlsanity.com/paper/2004.03066/) (2020)
- [Unveiling the Implicit Toxicity in Large Language Models](https://mlsanity.com/paper/2311.17391/) (2023)
- [Learning Implicit Sentiment in Aspect-based Sentiment Analysis with Supervised Contrastive Pre-Training](https://mlsanity.com/paper/2111.02194/) (2021)
- [R3: Reverse, Retrieve, and Rank for Sarcasm Generation with Commonsense Knowledge](https://mlsanity.com/paper/2004.13248/) (2020)
- [A Neural Pairwise Ranking Model for Readability Assessment](https://mlsanity.com/paper/2203.07450/) (2022)
- [Simplicity Level Estimate (SLE): A Learned Reference-Less Metric for Sentence Simplification](https://mlsanity.com/paper/2310.08170/) (2023)
- [Facilitating the Communication of Politeness through Fine-Grained Paraphrasing](https://mlsanity.com/paper/2012.00012/) (2020)
- [Conversational implicatures in English dialogue: Annotated dataset](https://mlsanity.com/paper/1911.10704/) (2019)

## Built on by (most cited first, 3 of 3)

- [Advancing Uto-Aztecan Language Technologies: A Case Study on the Endangered Comanche Language](https://mlsanity.com/paper/2505.18159/) (2025)
- [One Sentence, Two Embeddings: Contrastive Learning of Explicit and Implicit Semantic Representations](https://mlsanity.com/paper/2510.09293/) (2025)
- [Backdoor Learning in Language Models and Vision-Language Models](https://mlsanity.com/paper/2608.18095/) (2026)

## Abstract (arXiv)

Handling implicit language is essential for natural language processing systems to achieve precise text understanding and facilitate natural interactions with users. Despite its importance, the absence of a metric for accurately measuring the implicitness of language significantly constrains the depth of analysis possible in evaluating models' comprehension capabilities. This paper addresses this gap by developing a scalar metric that quantifies the implicitness level of language without relying on external references. Drawing on principles from traditional linguistics, we define "implicitness" as the divergence between semantic meaning and pragmatic interpretation. To operationalize this definition, we introduce ImpScore, a reference-free metric formulated through an interpretable regression model. This model is trained using pairwise contrastive learning on a specially curated dataset consisting of (implicit sentence, explicit sentence) pairs. We validate ImpScore through a user study that compares its assessments with human evaluations on out-of-distribution data, demonstrating its accuracy and strong correlation with human judgments. Additionally, we apply ImpScore to hate speech detection datasets, illustrating its utility and highlighting significant limitations in current large language models' ability to understand highly implicit content. Our metric is publicly available at https://github.com/audreycs/ImpScore.