# Confidence Modeling for Neural Semantic Parsing

arXiv: https://arxiv.org/abs/1805.04604 · HTML: https://mlsanity.com/paper/1805.04604/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2018.

## Built on by (most cited first, 30 of 33)

- [Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectly](https://mlsanity.com/paper/2204.13631/) (2022)
- [Selectively Answering Ambiguous Questions](https://mlsanity.com/paper/2305.14613/) (2023)
- [Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering](https://mlsanity.com/paper/2107.02331/) (2021)
- [Knowing More About Questions Can Help: Improving Calibration in Question Answering](https://mlsanity.com/paper/2106.01494/) (2021)
- [Calibrated Interpretation: Confidence Estimation in Semantic Parsing](https://mlsanity.com/paper/2211.07443/) (2022)
- [Learning to Reject with a Fixed Predictor: Application to Decontextualization](https://mlsanity.com/paper/2301.09044/) (2023)
- [A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications](https://mlsanity.com/paper/2111.01968/) (2021)
- [Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training](https://mlsanity.com/paper/2109.02284/) (2021)
- [Uncertainty-aware Language Modeling for Selective Question Answering](https://mlsanity.com/paper/2311.15451/) (2023)
- [Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?](https://mlsanity.com/paper/2402.16102/) (2024)
- [Calibration of Machine Reading Systems at Scale](https://mlsanity.com/paper/2203.10623/) (2022)
- [Confidence Estimation for Error Detection in Text-to-SQL Systems](https://mlsanity.com/paper/2501.09527/) (2025)
- [R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intents](https://mlsanity.com/paper/2303.00732/) (2023)
- [Toward Code Generation: A Survey and Lessons from Semantic Parsing](https://mlsanity.com/paper/2105.03317/) (2021)
- [Uncertainty Quantification in Retrieval Augmented Question Answering](https://mlsanity.com/paper/2502.18108/) (2025)
- [Learning to Insert [PAUSE] Tokens for Better Reasoning](https://mlsanity.com/paper/2506.03616/) (2025)
- [Uncertainty-Aware Bootstrap Learning for Joint Extraction on Distantly-Supervised Data](https://mlsanity.com/paper/2305.03827/) (2023)
- [Training Naturalized Semantic Parsers with Very Little Data](https://mlsanity.com/paper/2204.14243/) (2022)
- [Eeny, meeny, miny, moe. How to choose data for morphological inflection](https://mlsanity.com/paper/2210.14465/) (2022)
- [Know What I don't Know: Handling Ambiguous and Unanswerable Questions for Text-to-SQL](https://mlsanity.com/paper/2212.08902/) (2022)
- [The Case for Claim Difficulty Assessment in Automatic Fact Checking](https://mlsanity.com/paper/2109.09689/) (2021)
- [Pay More Attention to History: A Context Modelling Strategy for Conversational Text-to-SQL](https://mlsanity.com/paper/2112.08735/) (2021)
- [Calibrating LLMs for Text-to-SQL Parsing by Leveraging Sub-clause Frequencies](https://mlsanity.com/paper/2505.23804/) (2025)
- [Confidence over Time: Confidence Calibration with Temporal Logic for Large Language Model Reasoning](https://mlsanity.com/paper/2601.13387/) (2026)
- [Confidence-Guided Learning Process for Continuous Classification of Time Series](https://mlsanity.com/paper/2208.06883/) (2022)
- [A Formalism and Approach for Improving Robustness of Large Language Models Using Risk-Adjusted Confidence Scores](https://mlsanity.com/paper/2310.03283/) (2023)
- [Knowing When to Answer: Adaptive Confidence Refinement for Reliable Audio-Visual Question Answering](https://mlsanity.com/paper/2602.04924/) (2026)
- [Predicting generalization performance with correctness discriminators](https://mlsanity.com/paper/2311.09422/) (2023)
- [Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanism](https://mlsanity.com/paper/2402.12997/) (2024)
- [Equipping Transformer with Random-Access Reading for Long-Context Understanding](https://mlsanity.com/paper/2405.13216/) (2024)

## Abstract (arXiv)

In this work we focus on confidence modeling for neural semantic parsers which are built upon sequence-to-sequence models. We outline three major causes of uncertainty, and design various metrics to quantify these factors. These metrics are then used to estimate confidence scores that indicate whether model predictions are likely to be correct. Beyond confidence estimation, we identify which parts of the input contribute to uncertain predictions allowing users to interpret their model, and verify or refine its input. Experimental results show that our confidence model significantly outperforms a widely used method that relies on posterior probability, and improves the quality of interpretation compared to simply relying on attention scores.