# Probing Simile Knowledge from Pre-trained Language Models

arXiv: https://arxiv.org/abs/2204.12807 · HTML: https://mlsanity.com/paper/2204.12807/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 62 references that are in the map.
- First posted on arXiv 27 Apr 2022.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Factual Probing Is [MASK]: Learning vs. Learning to Recall](https://mlsanity.com/paper/2104.05240/) (2021)
- [Commonsense Knowledge Mining from Pretrained Models](https://mlsanity.com/paper/1909.00505/) (2019)
- [BERTese: Learning to Speak to BERT](https://mlsanity.com/paper/2103.05327/) (2021)
- [Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing](https://mlsanity.com/paper/2101.03289/) (2021)
- [MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding](https://mlsanity.com/paper/2103.06779/) (2021)
- [Metaphor Generation with Conceptual Mappings](https://mlsanity.com/paper/2106.01228/) (2021)
- [ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarization](https://mlsanity.com/paper/2202.05599/) (2022)
- [Neural Simile Recognition with Cyclic Multitask Learning and Local Attention](https://mlsanity.com/paper/1912.09084/) (2019)
- [Writing Polishment with Simile: Task, Dataset and A Neural Approach](https://mlsanity.com/paper/2012.08117/) (2020)
- ["Love is as Complex as Math": Metaphor Generation System for Social Chatbot](https://mlsanity.com/paper/2001.00733/) (2020)
- [Incorporating Commonsense Knowledge into Story Ending Generation via Heterogeneous Graph Networks](https://mlsanity.com/paper/2201.12538/) (2022)
- [Metaphor Interpretation Using Word Embeddings](https://mlsanity.com/paper/2010.02665/) (2020)

## Built on by (most cited first, 5 of 5)

- [Large Scale Knowledge Washing](https://mlsanity.com/paper/2405.16720/) (2024)
- [HAUSER: Towards Holistic and Automatic Evaluation of Simile Generation](https://mlsanity.com/paper/2306.07554/) (2023)
- [GOAL: Towards Benchmarking Few-Shot Sports Game Summarization](https://mlsanity.com/paper/2207.08635/) (2022)
- [MAPS-KB: A Million-scale Probabilistic Simile Knowledge Base](https://mlsanity.com/paper/2212.05254/) (2022)
- [I run as fast as a rabbit, can you? A Multilingual Simile Dialogue Dataset](https://mlsanity.com/paper/2306.05672/) (2023)

## Abstract (arXiv)

Simile interpretation (SI) and simile generation (SG) are challenging tasks for NLP because models require adequate world knowledge to produce predictions. Previous works have employed many hand-crafted resources to bring knowledge-related into models, which is time-consuming and labor-intensive. In recent years, pre-trained language models (PLMs) based approaches have become the de-facto standard in NLP since they learn generic knowledge from a large corpus. The knowledge embedded in PLMs may be useful for SI and SG tasks. Nevertheless, there are few works to explore it. In this paper, we probe simile knowledge from PLMs to solve the SI and SG tasks in the unified framework of simile triple completion for the first time. The backbone of our framework is to construct masked sentences with manual patterns and then predict the candidate words in the masked position. In this framework, we adopt a secondary training process (Adjective-Noun mask Training) with the masked language model (MLM) loss to enhance the prediction diversity of candidate words in the masked position. Moreover, pattern ensemble (PE) and pattern search (PS) are applied to improve the quality of predicted words. Finally, automatic and human evaluations demonstrate the effectiveness of our framework in both SI and SG tasks.