# A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering

arXiv: https://arxiv.org/abs/2106.07174 · HTML: https://mlsanity.com/paper/2106.07174/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 39 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing](https://mlsanity.com/paper/2009.13845/) (2020)
- [Simple and Effective Multi-Paragraph Reading Comprehension](https://mlsanity.com/paper/1710.10723/) (2017)
- [Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision](https://mlsanity.com/paper/1611.00020/) (2016)
- [A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization](https://mlsanity.com/paper/1902.01069/) (2019)
- [A Discrete Hard EM Approach for Weakly Supervised Question Answering](https://mlsanity.com/paper/1909.04849/) (2019)
- [MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension](https://mlsanity.com/paper/1905.13453/) (2019)
- [Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsing](https://mlsanity.com/paper/1807.02322/) (2018)
- [Quasar: Datasets for Question Answering by Search and Reading](https://mlsanity.com/paper/1707.03904/) (2017)
- [Learning to Generalize from Sparse and Underspecified Rewards](https://mlsanity.com/paper/1902.07198/) (2019)
- [Question Answering and Question Generation as Dual Tasks](https://mlsanity.com/paper/1706.02027/) (2017)
- [Hybrid Ranking Network for Text-to-SQL](https://mlsanity.com/paper/2008.04759/) (2020)
- [Semantic Parsing with Dual Learning](https://mlsanity.com/paper/1907.05343/) (2019)
- [Learning Semantic Parsers from Denotations with Latent Structured Alignments and Abstract Programs](https://mlsanity.com/paper/1909.04165/) (2019)
- [Densely Connected Attention Propagation for Reading Comprehension](https://mlsanity.com/paper/1811.04210/) (2018)
- [Jointly Learning Semantic Parser and Natural Language Generator via Dual Information Maximization](https://mlsanity.com/paper/1906.00575/) (2019)
- [Weakly-supervised Neural Semantic Parsing with a Generative Ranker](https://mlsanity.com/paper/1808.07625/) (2018)
- [Multi-Mention Learning for Reading Comprehension with Neural Cascades](https://mlsanity.com/paper/1711.00894/) (2017)

## Built on by (most cited first, 2 of 2)

- [Chaining Simultaneous Thoughts for Numerical Reasoning](https://mlsanity.com/paper/2211.16482/) (2022)
- [NAPG: Non-Autoregressive Program Generation for Hybrid Tabular-Textual Question Answering](https://mlsanity.com/paper/2211.03462/) (2022)

## Abstract (arXiv)

Weakly supervised question answering usually has only the final answers as supervision signals while the correct solutions to derive the answers are not provided. This setting gives rise to the spurious solution problem: there may exist many spurious solutions that coincidentally derive the correct answer, but training on such solutions can hurt model performance (e.g., producing wrong solutions or answers). For example, for discrete reasoning tasks as on DROP, there may exist many equations to derive a numeric answer, and typically only one of them is correct. Previous learning methods mostly filter out spurious solutions with heuristics or using model confidence, but do not explicitly exploit the semantic correlations between a question and its solution. In this paper, to alleviate the spurious solution problem, we propose to explicitly exploit such semantic correlations by maximizing the mutual information between question-answer pairs and predicted solutions. Extensive experiments on four question answering datasets show that our method significantly outperforms previous learning methods in terms of task performance and is more effective in training models to produce correct solutions.