Computational models of pragmatic reasoning with flexible generation of meaning and expression alternatives
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Pragmatic language use requires reasoning about alternatives: the alternative expressions a speaker might have chosen, or the alternative interpretations a listener might entertain. Formal and computational models of pragmatics must therefore specify the sets of alternatives that interlocutors reason over, which is often done through manual specification. Here we propose a framework, ScAffolded Generative models for Explanation (SAGE), that combines the explanatory transparency of cognitive models with the generative flexibility of language models (LMs). SAGE decomposes a pragmatic process into three kinds of modules: proposers, which use LMs to generate an open-ended space of candidate alternatives; evaluators, which assess those alternatives (e.g., their semantics, complexity, or typicality); and selectors, which implement the rule-based computational steps of a cognitively motivated task analysis. We assess SAGE in three case studies spanning pragmatic generation and interpretation-referential expression generation, manner (M-)implicatures, and Gricean conversational implicatures. SAGE models are evaluated critically using established methods from computational cognitive modeling, including ablations, baseline comparisons, and quantitative fit to human data. Across studies, SAGE models achieved high accuracy and often outperformed baselines, but component-level analyses reveal an asymmetry: LM proposers reliably generated alternatives well-suited to pragmatic modeling, whereas LM evaluators are better at providing intuitive judgements rather than judgements of theoretical or formal measures. We discuss the promise and the limitations of neuro-symbolic models as candidate explanatory accounts of human pragmatic language use.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 23 of its 125 references that are in the map. First posted on arXiv 20 Jul 2026.
Builds on
23 of 125 references.
Closest ancestors · 5
Same direction, newest first
- 2023Turning large language models into cognitive models6 Jun
- 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-47 Apr
- 2023LLaMA: Open and Efficient Foundation Language Models27 Feb
- 2022PAL: Program-aided Language Models18 Nov
- 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning19 May
Other references · 18
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
- 2019Language Models as Knowledge Bases?cited by 1,922
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2023Cognitive Architectures for Language Agentscited by 422
- 2023ART: Automatic multi-step reasoning and tool-use for large language modelscited by 193
- 2024LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworkscited by 178
- 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
- 2023Solving Math Word Problems by Combining Language Models With Symbolic Solverscited by 124
6 more
- 2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 50
- 2024Centaur: a foundation model of human cognitioncited by 34
- 2025Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Modelscited by 14
- 2025Non-literal Understanding of Number Words by Language Modelscited by 9
- 2025Using LLMs to Advance the Cognitive Science of Collectivescited by 6
- 2024Cognitive Modeling with Scaffolded LLMs: A Case Study of Referential Expression Generationcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025On the Same Wavelength? Evaluating Pragmatic Reasoning in Language Models across Broad Concepts
- 2025Pragmatic Theories Enhance Understanding of Implied Meanings in LLMs
- 2025The Pragmatic Mind of Machines: Tracing the Emergence of Pragmatic Competence in Large Language Models
- 2025Understand the Implication: Learning to Think for Pragmatic Understanding
- 2026Evaluating Pragmatic Reasoning in Large Language Models: Evidence from Scalar Diversity
- 2026CEI: A Benchmark for Evaluating Pragmatic Reasoning in Language Models
- 2024Towards an Analysis of Discourse and Interactional Pragmatic Reasoning Capabilities of Large Language Models
- 2026Which one is banana man? Evaluating vision-language models in multi-turn pragmatic interpretation