# Unveiling the Limits of Large Language Models in Inferring Pragmatic Meaning from Non-Verbal Responses

arXiv: https://arxiv.org/abs/2606.01845 · HTML: https://mlsanity.com/paper/2606.01845/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 44 references that are in the map.
- First posted on arXiv 1 Jun 2026.

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm](https://mlsanity.com/paper/1708.00524/) (2017)
- [MuTual: A Dataset for Multi-Turn Dialogue Reasoning](https://mlsanity.com/paper/2004.04494/) (2020)
- [Overthinking the Truth: Understanding how Language Models Process False Demonstrations](https://mlsanity.com/paper/2307.09476/) (2023)
- [A fine-grained comparison of pragmatic language understanding in humans and language models](https://mlsanity.com/paper/2212.06801/) (2022)
- [The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMs](https://mlsanity.com/paper/2210.14986/) (2022)
- [Generalization Bias in Large Language Model Summarization of Scientific Research](https://mlsanity.com/paper/2504.00025/) (2025)
- [Pragmatic Metacognitive Prompting Improves LLM Performance on Sarcasm Detection](https://mlsanity.com/paper/2412.04509/) (2024)
- [Nonverbal Cues in Human-Robot Interaction: A Communication Studies Perspective](https://mlsanity.com/paper/2304.11293/) (2023)
- [Do Large Language Models Understand Conversational Implicature -- A case study with a chinese sitcom](https://mlsanity.com/paper/2404.19509/) (2024)
- [Spice up Your Chat: The Intentions and Sentiment Effects of Using Emoji](https://mlsanity.com/paper/1703.02860/) (2017)
- [Controlled Cue Generation for Play Scripts](https://mlsanity.com/paper/2112.06953/) (2021)

## Built on by (most cited first, 1 of 1)

- [Pragmatic Attack Surface: Vulnerabilities of Implicit Context in Large Language Models](https://mlsanity.com/paper/2608.09551/) (2026)

## Abstract (arXiv)

Although large language models (LLMs) have shown considerable progress in pragmatic language understanding, prior research has focused mainly on their comprehension of verbal behavior. Nonetheless, non-verbal behavior remains a fundamental component of human communication, especially when deliberately utilized in isolation to convey indirect meanings. In this work, we present the first systematic evaluation of LLMs' ability to infer pragmatic meaning in dialogue consisting solely of non-verbal responses. We explore three research questions: (1) Can LLMs recognize indirect intent conveyed through non-verbal responses? (2) When and how do LLMs fail to capture non-verbal intent? (3) How can we improve LLMs' ability to interpret non-verbal intent?. Through the evaluation, we observe that LLMs struggle to infer underlying meaning from non-verbal responses, with accuracy dropping by up to 60% points compared to verbal ones. Further extensive analysis reveals a behavioral pattern in LLMs' interpretations of non-verbal behavior and demonstrates that in-context learning facilitates pragmatic inference.