# Rethinking Label Smoothing on Multi-hop Question Answering

arXiv: https://arxiv.org/abs/2212.09512 · HTML: https://mlsanity.com/paper/2212.09512/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 45 references that are in the map.
- First posted on arXiv 19 Dec 2022.

## Builds on (most cited first, 30 of 30)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [Does label smoothing mitigate label noise?](https://mlsanity.com/paper/2003.02819/) (2020)
- [Constructing Datasets for Multi-hop Reading Comprehension Across Documents](https://mlsanity.com/paper/1710.06481/) (2017)
- [Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering](https://mlsanity.com/paper/1911.10470/) (2019)
- [Towards better decoding and language model integration in sequence to sequence models](https://mlsanity.com/paper/1612.02695/) (2016)
- [Hierarchical Graph Network for Multi-hop Question Answering](https://mlsanity.com/paper/1911.03631/) (2019)
- [Multi-hop Question Answering](https://mlsanity.com/paper/2204.09140/) (2022)
- [Dynamically Fused Graph Network for Multi-hop Reasoning](https://mlsanity.com/paper/1905.06933/) (2019)
- [Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents](https://mlsanity.com/paper/1911.00484/) (2019)
- [Multi-hop Question Answering via Reasoning Chains](https://mlsanity.com/paper/1910.02610/) (2019)
- [Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction](https://mlsanity.com/paper/1905.08511/) (2019)
- [Multi-Hop Paragraph Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/1906.06606/) (2019)
- [Regularization via Structural Label Smoothing](https://mlsanity.com/paper/2001.01900/) (2020)
- [A Simple Yet Strong Pipeline for HotpotQA](https://mlsanity.com/paper/2004.06753/) (2020)
- [Boundary Smoothing for Named Entity Recognition](https://mlsanity.com/paper/2204.12031/) (2022)
- [Generalizing Back-Translation in Neural Machine Translation](https://mlsanity.com/paper/1906.07286/) (2019)
- [Towards Understanding Label Smoothing](https://mlsanity.com/paper/2006.11653/) (2020)
- [Semantic Label Smoothing for Sequence to Sequence Problems](https://mlsanity.com/paper/2010.07447/) (2020)
- [Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy](https://mlsanity.com/paper/2107.11823/) (2021)
- [From Easy to Hard: Two-stage Selector and Reader for Multi-hop Question Answering](https://mlsanity.com/paper/2205.11729/) (2022)
- [Weakly Supervised Label Smoothing](https://mlsanity.com/paper/2012.08575/) (2020)
- [Similarity Based Label Smoothing For Dialogue Generation](https://mlsanity.com/paper/2107.11481/) (2021)

## Built on by (most cited first, 4 of 4)

- [End-to-End Beam Retrieval for Multi-Hop Question Answering](https://mlsanity.com/paper/2308.08973/) (2023)
- [CRUSH4SQL: Collective Retrieval Using Schema Hallucination For Text2SQL](https://mlsanity.com/paper/2311.01173/) (2023)
- [Gumbel Reranking: Differentiable End-to-End Reranker Optimization](https://mlsanity.com/paper/2502.11116/) (2025)
- [Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Tasks](https://mlsanity.com/paper/2501.06286/) (2025)

## Abstract (arXiv)

Multi-Hop Question Answering (MHQA) is a significant area in question answering, requiring multiple reasoning components, including document retrieval, supporting sentence prediction, and answer span extraction. In this work, we analyze the primary factors limiting the performance of multi-hop reasoning and introduce label smoothing into the MHQA task. This is aimed at enhancing the generalization capabilities of MHQA systems and mitigating overfitting of answer spans and reasoning paths in training set. We propose a novel label smoothing technique, F1 Smoothing, which incorporates uncertainty into the learning process and is specifically tailored for Machine Reading Comprehension (MRC) tasks. Inspired by the principles of curriculum learning, we introduce the Linear Decay Label Smoothing Algorithm (LDLA), which progressively reduces uncertainty throughout the training process. Experiment on the HotpotQA dataset demonstrates the effectiveness of our methods in enhancing performance and generalizability in multi-hop reasoning, achieving new state-of-the-art results on the leaderboard.