# Fine-grained Contrastive Learning for Definition Generation

arXiv: https://arxiv.org/abs/2210.00543 · HTML: https://mlsanity.com/paper/2210.00543/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 2 Oct 2022.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation](https://mlsanity.com/paper/2109.05729/) (2021)
- [Definition Modeling: Learning to define word embeddings in natural language](https://mlsanity.com/paper/1612.00394/) (2016)
- [PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation](https://mlsanity.com/paper/2004.07159/) (2020)
- [Conditional Generators of Words Definitions](https://mlsanity.com/paper/1806.10090/) (2018)
- [Learning to Explain Non-Standard English Words and Phrases](https://mlsanity.com/paper/1709.09254/) (2017)
- [Multitasking Framework for Unsupervised Simple Definition Generation](https://mlsanity.com/paper/2203.12926/) (2022)
- [Incorporating Sememes into Chinese Definition Modeling](https://mlsanity.com/paper/1905.06512/) (2019)
- [VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling](https://mlsanity.com/paper/2010.03124/) (2020)
- [pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference](https://mlsanity.com/paper/1810.08854/) (2018)

## Built on by (most cited first, 10 of 10)

- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [Multi-level Contrastive Learning for Script-based Character Understanding](https://mlsanity.com/paper/2310.13231/) (2023)
- [Assisting Language Learners: Automated Trans-Lingual Definition Generation via Contrastive Prompt Learning](https://mlsanity.com/paper/2306.06058/) (2023)
- [A Question-centric Multi-experts Contrastive Learning Framework for Improving the Accuracy and Interpretability of Deep Sequential Knowledge Tracing Models](https://mlsanity.com/paper/2403.07322/) (2024)
- [Understanding Multimodal Deep Neural Networks: A Concept Selection View](https://mlsanity.com/paper/2404.08964/) (2024)
- [Improving Low-Resource Knowledge Tracing Tasks by Supervised Pre-training and Importance Mechanism Fine-tuning](https://mlsanity.com/paper/2403.06725/) (2024)
- [Balancing Speciality and Versatility: A Coarse to Fine Framework for Mitigating Catastrophic Forgetting in Large Language Models](https://mlsanity.com/paper/2404.10306/) (2024)
- [SLM4Offer: Personalized Marketing Offer Generation Using Contrastive Learning Based Fine-Tuning](https://mlsanity.com/paper/2508.15471/) (2025)
- [LM-Lexicon: Improving Definition Modeling via Harmonizing Semantic Experts](https://mlsanity.com/paper/2602.14060/) (2026)
- [SemBench: A Universal Semantic Framework for LLM Evaluation](https://mlsanity.com/paper/2603.11687/) (2026)

## Abstract (arXiv)

Recently, pre-trained transformer-based models have achieved great success in the task of definition generation (DG). However, previous encoder-decoder models lack effective representation learning to contain full semantic components of the given word, which leads to generating under-specific definitions. To address this problem, we propose a novel contrastive learning method, encouraging the model to capture more detailed semantic representations from the definition sequence encoding. According to both automatic and manual evaluation, the experimental results on three mainstream benchmarks demonstrate that the proposed method could generate more specific and high-quality definitions compared with several state-of-the-art models.