# Lightweight Decoding Strategies for Increasing Specificity

arXiv: https://arxiv.org/abs/2110.11850 · HTML: https://mlsanity.com/paper/2110.11850/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 22 Oct 2021.

## Builds on (most cited first, 8 of 8)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization](https://mlsanity.com/paper/1809.05972/) (2018)
- [Controllable Abstractive Summarization](https://mlsanity.com/paper/1711.05217/) (2017)
- [Domain Agnostic Real-Valued Specificity Prediction](https://mlsanity.com/paper/1811.05085/) (2018)
- [An Attentional Neural Conversation Model with Improved Specificity](https://mlsanity.com/paper/1606.01292/) (2016)

## Abstract (arXiv)

Language models are known to produce vague and generic outputs. We propose two unsupervised decoding strategies based on either word-frequency or point-wise mutual information to increase the specificity of any model that outputs a probability distribution over its vocabulary at generation time. We test the strategies in a prompt completion task; with human evaluations, we find that both strategies increase the specificity of outputs with only modest decreases in sensibility. We also briefly present a summarization use case, where these strategies can produce more specific summaries.