# Can Large Language Models Understand Internet Buzzwords Through User-Generated Content

arXiv: https://arxiv.org/abs/2505.15071 · HTML: https://mlsanity.com/paper/2505.15071/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 72 references that are in the map.
- First posted on arXiv 21 May 2025.

## Builds on (most cited first, 20 of 20)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets](https://mlsanity.com/paper/2307.10928/) (2023)
- [Definition Modeling: Learning to define word embeddings in natural language](https://mlsanity.com/paper/1612.00394/) (2016)
- [Evaluating Mixed-initiative Conversational Search Systems via User Simulation](https://mlsanity.com/paper/2204.08046/) (2022)
- [Multitasking Framework for Unsupervised Simple Definition Generation](https://mlsanity.com/paper/2203.12926/) (2022)
- [Mark my Word: A Sequence-to-Sequence Approach to Definition Modeling](https://mlsanity.com/paper/1911.05715/) (2019)
- [Understanding Jargon: Combining Extraction and Generation for Definition Modeling](https://mlsanity.com/paper/2111.07267/) (2021)
- [Incorporating Sememes into Chinese Definition Modeling](https://mlsanity.com/paper/1905.06512/) (2019)
- [Assisting Language Learners: Automated Trans-Lingual Definition Generation via Contrastive Prompt Learning](https://mlsanity.com/paper/2306.06058/) (2023)
- [LLMs' Understanding of Natural Language Revealed](https://mlsanity.com/paper/2407.19630/) (2024)
- [Legend: Leveraging Representation Engineering to Annotate Safety Margin for Preference Datasets](https://mlsanity.com/paper/2406.08124/) (2024)
- [Can large language models understand uncommon meanings of common words?](https://mlsanity.com/paper/2405.05741/) (2024)
- [Towards Equipping Transformer with the Ability of Systematic Compositionality](https://mlsanity.com/paper/2312.07280/) (2023)
- [Candidate sentence selection for language learning exercises: from a comprehensive framework to an empirical evaluation](https://mlsanity.com/paper/1706.03530/) (2017)

## Built on by (most cited first, 2 of 2)

- [Beyond Literal Translation: Evaluating Cultural Effectiveness in Social Media UGC](https://mlsanity.com/paper/2605.25626/) (2026)
- [Towards High-Level Semantic Intelligence](https://mlsanity.com/paper/2607.24082/) (2026)

## Abstract (arXiv)

The massive user-generated content (UGC) available in Chinese social media is giving rise to the possibility of studying internet buzzwords. In this paper, we study if large language models (LLMs) can generate accurate definitions for these buzzwords based on UGC as examples. Our work serves a threefold contribution. First, we introduce CHEER, the first dataset of Chinese internet buzzwords, each annotated with a definition and relevant UGC. Second, we propose a novel method, called RESS, to effectively steer the comprehending process of LLMs to produce more accurate buzzword definitions, mirroring the skills of human language learning. Third, with CHEER, we benchmark the strengths and weaknesses of various off-the-shelf definition generation methods and our RESS. Our benchmark demonstrates the effectiveness of RESS while revealing crucial shared challenges: over-reliance on prior exposure, underdeveloped inferential abilities, and difficulty identifying high-quality UGC to facilitate comprehension. We believe our work lays the groundwork for future advancements in LLM-based definition generation. Our dataset and code are available at https://github.com/SCUNLP/Buzzword.