mlsanityData to 14 Oct 2026

Ambiguity in LLMs is a concept missing problem

arXiv 2505.11679 ↗First posted 16 May 2025cs.CL cs.LGcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Ambiguity in natural language is a significant obstacle for achieving accurate text to structured data mapping through large language models (LLMs), which affects the performance of tasks such as mapping text to agentic tool calling and text-to-SQL queries. Existing methods to ambiguity handling either rely on the ReACT framework to obtain correct mappings through trial and error, or on supervised fine-tuning to bias models toward specific tasks. In this paper, we adopt a different approach that characterizes representation differences of ambiguous text in the latent space and leverages these differences to identify ambiguity before mapping them to structured data. To detect sentence-level ambiguity, we focus on the relationship between ambiguous questions and their interpretations. Unlike distances calculated by dense embeddings, we introduce a new distance measure based on a path kernel over concepts. With this measurement, we identify patterns to distinguish ambiguous from unambiguous questions. Furthermore, we propose a method for improving LLM performance on ambiguous agentic tool calling through missing concept prediction. Both achieve state-of-the-art results.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 17 of its 26 references that are in the map. First posted on arXiv 16 May 2025.

Builds on

17 of 26 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Aligning Language Models to Explicitly Handle Ambiguity18 Apr

Other references · 16

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  3. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  4. 2022Toy Models of Superpositioncited by 971
  5. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  6. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  7. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
  8. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 234
  9. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 126
  10. 2020Every Model Learned by Gradient Descent Is Approximately a Kernel Machinecited by 55
  11. 2024AMBROSIA: A Benchmark for Parsing Ambiguous Questions into Database Queriescited by 35
  12. 2023Interactive-Chain-Prompting: Ambiguity Resolution for Crosslingual Conditional Generation with Interactioncited by 16
4 more
  1. 2023Zero and Few-shot Semantic Parsing with Ambiguous Inputscited by 16
  2. 2023Answering Ambiguous Questions via Iterative Promptingcited by 13
  3. 2025Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsingcited by 13
  4. 2024Scope Ambiguities in Large Language Modelscited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Survey
  2. 2024CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Models
  3. 2026From Answers to Interpretations: Rethinking Ambiguity-Induced Aleatoric Uncertainty Estimation in LLMs
  4. 2025Agent-Based Detection and Resolution of Incompleteness and Ambiguity in Interactions with Large Language Models
  5. 2025Active Task Disambiguation with LLMs
  6. 2026Small Language Model Helps Resolve Semantic Ambiguity of LLM Prompt
  7. 2023Reasoning about Ambiguous Definite Descriptions
  8. 2026A Tri-Agent Framework for Evaluating and Aligning Question Clarification Capabilities of Large Language Models