# Is It JUST Semantics? A Case Study of Discourse Particle Understanding in LLMs

arXiv: https://arxiv.org/abs/2506.04534 · HTML: https://mlsanity.com/paper/2506.04534/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 9 of 9)

- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [A Systematic Assessment of Syntactic Generalization in Neural Language Models](https://mlsanity.com/paper/2005.03692/) (2020)
- [minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models](https://mlsanity.com/paper/2203.13112/) (2022)
- [Probing What Different NLP Tasks Teach Machines about Function Word Comprehension](https://mlsanity.com/paper/1904.11544/) (2019)
- [Pragmatic competence of pre-trained language models through the lens of discourse connectives](https://mlsanity.com/paper/2109.12951/) (2021)
- [Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Models](https://mlsanity.com/paper/2105.03495/) (2021)
- [Are LLMs Good Annotators for Discourse-level Event Relation Extraction?](https://mlsanity.com/paper/2407.19568/) (2024)

## Built on by (most cited first, 4 of 4)

- [Accommodation and Epistemic Vigilance: A Pragmatic Account of Why LLMs Fail to Challenge Harmful Beliefs](https://mlsanity.com/paper/2601.04435/) (2026)
- [Can Large Language Models Handle Discourse Particles? A Case Study of Colloquial Malay](https://mlsanity.com/paper/2605.28782/) (2026)
- [Which course? Discourse! Teaching Discourse and Generation in the Era of LLMs](https://mlsanity.com/paper/2602.02878/) (2026)
- [Not Just an Evaluation: A Case Study of Discourse Particle Sensitivity in Language Models](https://mlsanity.com/paper/2610.13794/) (2026)

## Abstract (arXiv)

Discourse particles are crucial elements that subtly shape the meaning of text. These words, often polyfunctional, give rise to nuanced and often quite disparate semantic/discourse effects, as exemplified by the diverse uses of the particle "just" (e.g., exclusive, temporal, emphatic). This work investigates the capacity of LLMs to distinguish the fine-grained senses of English "just", a well-studied example in formal semantics, using data meticulously created and labeled by expert linguists. Our findings reveal that while LLMs exhibit some ability to differentiate between broader categories, they struggle to fully capture more subtle nuances, highlighting a gap in their understanding of discourse particles.