# Zero and Few-shot Semantic Parsing with Ambiguous Inputs

arXiv: https://arxiv.org/abs/2306.00824 · HTML: https://mlsanity.com/paper/2306.00824/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 65 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 29 of 29)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [Augmented Language Models: a Survey](https://mlsanity.com/paper/2302.07842/) (2023)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [COGS: A Compositional Generalization Challenge Based on Semantic Interpretation](https://mlsanity.com/paper/2010.05465/) (2020)
- [TALM: Tool Augmented Language Models](https://mlsanity.com/paper/2205.12255/) (2022)
- [Constrained Language Models Yield Few-Shot Semantic Parsers](https://mlsanity.com/paper/2104.08768/) (2021)
- [Language to Logical Form with Neural Attention](https://mlsanity.com/paper/1601.01280/) (2016)
- [What Can We Learn from Collective Human Opinions on Natural Language Inference Data?](https://mlsanity.com/paper/2010.03532/) (2020)
- [We're Afraid Language Models Aren't Modeling Ambiguity](https://mlsanity.com/paper/2304.14399/) (2023)
- [Few-Shot Semantic Parsing with Language Models Trained On Code](https://mlsanity.com/paper/2112.08696/) (2021)
- [Uncertain Natural Language Inference](https://mlsanity.com/paper/1909.03042/) (2019)
- [Why Does a Visual Question Have Different Answers?](https://mlsanity.com/paper/1908.04342/) (2019)
- [Calibrated Interpretation: Confidence Estimation in Semantic Parsing](https://mlsanity.com/paper/2211.07443/) (2022)
- [Broad-Coverage Semantic Parsing as Transduction](https://mlsanity.com/paper/1909.02607/) (2019)
- [Efficient Online Scalar Annotation with Bounded Support](https://mlsanity.com/paper/1806.01170/) (2018)
- [Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation](https://mlsanity.com/paper/2212.10140/) (2022)
- [ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretation](https://mlsanity.com/paper/2303.13716/) (2023)
- [Fine-Grained Temporal Relation Extraction](https://mlsanity.com/paper/1902.01390/) (2019)
- [Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA](https://mlsanity.com/paper/2211.07516/) (2022)
- [AmbiCoref: Evaluating Human and Model Sensitivity to Ambiguous Coreference](https://mlsanity.com/paper/2302.00762/) (2023)
- [Do You See What I Mean? Visual Resolution of Linguistic Ambiguities](https://mlsanity.com/paper/1603.08079/) (2016)
- [Practical Semantic Parsing for Spoken Language Understanding](https://mlsanity.com/paper/1903.04521/) (2019)
- [Is the Elephant Flying? Resolving Ambiguities in Text-to-Image Generative Models](https://mlsanity.com/paper/2211.12503/) (2022)
- [Did You Mean...? Confidence-based Trade-offs in Semantic Parsing](https://mlsanity.com/paper/2303.16857/) (2023)

## Built on by (most cited first, 16 of 16)

- [Retrieval-Augmented Generation with Conflicting Evidence](https://mlsanity.com/paper/2504.13079/) (2025)
- [Uncertainty in Natural Language Generation: From Theory to Applications](https://mlsanity.com/paper/2307.15703/) (2023)
- [Grammar-Aligned Decoding](https://mlsanity.com/paper/2405.21047/) (2024)
- [AMBROSIA: A Benchmark for Parsing Ambiguous Questions into Database Queries](https://mlsanity.com/paper/2406.19073/) (2024)
- [Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective](https://mlsanity.com/paper/2506.05754/) (2025)
- [Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsing](https://mlsanity.com/paper/2502.18448/) (2025)
- [Scope Ambiguities in Large Language Models](https://mlsanity.com/paper/2404.04332/) (2024)
- [Do Pre-Trained Language Models Detect and Understand Semantic Underspecification? Ask the DUST!](https://mlsanity.com/paper/2402.12486/) (2024)
- [Text-to-SQL Calibration: No Need to Ask -- Just Rescale Model Probabilities](https://mlsanity.com/paper/2411.16742/) (2024)
- [Superlatives in Context: Modeling the Implicit Semantics of Superlatives](https://mlsanity.com/paper/2405.20967/) (2024)
- [Trick or Neat: Adversarial Ambiguity and Language Model Evaluation](https://mlsanity.com/paper/2506.01205/) (2025)
- [Unified Multimodal Uncertain Inference](https://mlsanity.com/paper/2604.08701/) (2026)
- [Language Models Identify Ambiguities and Exploit Loopholes](https://mlsanity.com/paper/2508.19546/) (2025)
- [Ambiguity in LLMs is a concept missing problem](https://mlsanity.com/paper/2505.11679/) (2025)
- [Can LLMs Detect Ambiguous Plural Reference? An Analysis of Split-Antecedent and Mereological Reference](https://mlsanity.com/paper/2510.04581/) (2025)
- [Small Language Model Helps Resolve Semantic Ambiguity of LLM Prompt](https://mlsanity.com/paper/2604.23263/) (2026)

## Abstract (arXiv)

Despite the frequent challenges posed by ambiguity when representing meaning via natural language, it is often ignored or deliberately removed in tasks mapping language to formally-designed representations, which generally assume a one-to-one mapping between linguistic and formal representations. We attempt to address this shortcoming by introducing AmP, a framework, dataset, and challenge for translating ambiguous natural language to formal representations like logic and code. We define templates and generate data for five well-documented linguistic ambiguities. Using AmP, we investigate how several few-shot text-to-code systems handle ambiguity, introducing three new metrics. We find that large pre-trained models perform poorly at capturing the distribution of possible meanings without deliberate instruction. However, models are able to capture the distribution well when ambiguity is attested in their inputs. These results motivate a call for including ambiguity explicitly in datasets and promote considering the distribution of possible outputs when evaluating systems. Data and code: https://github.com/esteng/ambiguous_parsing