# False Friends in the Shell: Unveiling the Emoticon Semantic Confusion in Large Language Models

arXiv: https://arxiv.org/abs/2601.07885 · HTML: https://mlsanity.com/paper/2601.07885/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 49 references that are in the map.
- First posted on arXiv 12 Jan 2026.

## Builds on (most cited first, 15 of 15)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [MART: Improving LLM Safety with Multi-round Automatic Red-Teaming](https://mlsanity.com/paper/2311.07689/) (2023)
- [Red-Teaming LLM Multi-Agent Systems via Communication Attacks](https://mlsanity.com/paper/2502.14847/) (2025)
- [Sentiment of Emojis](https://mlsanity.com/paper/1509.07761/) (2015)
- [Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo Chat](https://mlsanity.com/paper/2401.17163/) (2024)
- [Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detection](https://mlsanity.com/paper/2411.01077/) (2024)
- [An Optimizable Suffix Is Worth A Thousand Templates: Efficient Black-box Jailbreaking without Affirmative Phrases via LLM as Optimizer](https://mlsanity.com/paper/2408.11313/) (2024)
- [Linguistic Characteristics of AI-Generated Text: A Survey](https://mlsanity.com/paper/2510.05136/) (2025)
- [X-TURING: Towards an Enhanced and Efficient Turing Test for Long-Term Dialogue Agents](https://mlsanity.com/paper/2408.09853/) (2024)
- [Irony in Emojis: A Comparative Study of Human and LLM Interpretation](https://mlsanity.com/paper/2501.11241/) (2025)
- [A First Look at Emoji Usage on GitHub: An Empirical Study](https://mlsanity.com/paper/1812.04863/) (2018)
- [ABFS: Natural Robustness Testing for LLM-based NLP Software](https://mlsanity.com/paper/2503.01319/) (2025)

## Built on by (most cited first, 1 of 1)

- [Overeager Coding Agents: Measuring Out-of-Scope Actions on Benign Tasks](https://mlsanity.com/paper/2605.18583/) (2026)

## Abstract (arXiv)

Emoticons are widely used in digital communication to convey affective intent, yet their safety implications for Large Language Models (LLMs) remain largely unexplored. In this paper, we identify emoticon semantic confusion, a vulnerability where LLMs misinterpret ASCII-based emoticons to perform unintended and even destructive actions. To systematically study this phenomenon, we develop an automated data generation pipeline and construct a dataset containing 3,757 code-oriented test cases spanning 21 meta-scenarios, four programming languages, and varying contextual complexities. Our study on six LLMs reveals that emoticon semantic confusion is pervasive, with an average confusion ratio exceeding 38%. More critically, over 90% of confused responses yield 'silent failures', which are syntactically valid outputs but deviate from user intent, potentially leading to destructive security consequences. Furthermore, we observe that this vulnerability readily transfers to popular agent frameworks, while existing prompt-based mitigations remain largely ineffective. We call on the community to recognize this emerging vulnerability and develop effective mitigation methods to uphold the safety and reliability of the LLM system.