# Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Texts

arXiv: https://arxiv.org/abs/2210.16865 · HTML: https://mlsanity.com/paper/2210.16865/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 39 references that are in the map.
- First posted on arXiv 30 Oct 2022.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [The Web as a Knowledge-base for Answering Complex Questions](https://mlsanity.com/paper/1803.06643/) (2018)
- [PAWS: Paraphrase Adversaries from Word Scrambling](https://mlsanity.com/paper/1904.01130/) (2019)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://mlsanity.com/paper/1610.02424/) (2016)
- [Constrained Language Models Yield Few-Shot Semantic Parsers](https://mlsanity.com/paper/2104.08768/) (2021)
- [Break It Down: A Question Understanding Benchmark](https://mlsanity.com/paper/2001.11770/) (2020)
- ["Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understanding](https://mlsanity.com/paper/1909.03065/) (2019)
- [Multi-hop Reading Comprehension through Question Decomposition and Rescoring](https://mlsanity.com/paper/1906.02916/) (2019)
- [Unsupervised Question Decomposition for Question Answering](https://mlsanity.com/paper/2002.09758/) (2020)
- [Neural Module Networks for Reasoning over Text](https://mlsanity.com/paper/1912.04971/) (2019)
- [TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions](https://mlsanity.com/paper/2005.00242/) (2020)
- [Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models](https://mlsanity.com/paper/2009.00751/) (2020)
- [Temporal Reasoning on Implicit Events from Distant Supervision](https://mlsanity.com/paper/2010.12753/) (2020)
- [Temporal Common Sense Acquisition with Minimal Supervision](https://mlsanity.com/paper/2005.04304/) (2020)
- [Challenges in Generalization in Open Domain Question Answering](https://mlsanity.com/paper/2109.01156/) (2021)
- [Incidental Supervision: Moving beyond Supervised Learning](https://mlsanity.com/paper/2005.12339/) (2020)
- [Hey AI, Can You Solve Complex Tasks by Talking to Agents?](https://mlsanity.com/paper/2110.08542/) (2021)

## Built on by (most cited first, 18 of 18)

- [To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning](https://mlsanity.com/paper/2409.12183/) (2024)
- [A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners](https://mlsanity.com/paper/2406.11050/) (2024)
- [Natural Language Reasoning, A Survey](https://mlsanity.com/paper/2303.14725/) (2023)
- [Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representations](https://mlsanity.com/paper/2311.04335/) (2023)
- [Generic Temporal Reasoning with Differential Analysis and Explanation](https://mlsanity.com/paper/2212.10467/) (2022)
- [BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models](https://mlsanity.com/paper/2404.12494/) (2024)
- [Iterated Decomposition: Improving Science Q&A by Supervising Reasoning Processes](https://mlsanity.com/paper/2301.01751/) (2023)
- [DORIS-MAE: Scientific Document Retrieval using Multi-level Aspect-based Queries](https://mlsanity.com/paper/2310.04678/) (2023)
- [Large Language Models are Contrastive Reasoners](https://mlsanity.com/paper/2403.08211/) (2024)
- [Retrieval-Enhanced Machine Learning: Synthesis and Opportunities](https://mlsanity.com/paper/2407.12982/) (2024)
- [Question Decomposition for Retrieval-Augmented Generation](https://mlsanity.com/paper/2507.00355/) (2025)
- [DialCoT Meets PPO: Decomposing and Exploring Reasoning Paths in Smaller Language Models](https://mlsanity.com/paper/2310.05074/) (2023)
- [POQD: Performance-Oriented Query Decomposer for Multi-vector retrieval](https://mlsanity.com/paper/2505.19189/) (2025)
- [DEXTER: A Benchmark for open-domain Complex Question Answering using LLMs](https://mlsanity.com/paper/2406.17158/) (2024)
- [RECAP: Transparent Inference-Time Emotion Alignment for Medical Dialogue Systems](https://mlsanity.com/paper/2509.10746/) (2025)
- [Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability](https://mlsanity.com/paper/2504.16056/) (2025)
- [Building Interpretable and Reliable Open Information Retriever for New Domains Overnight](https://mlsanity.com/paper/2308.04756/) (2023)
- [Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Tasks](https://mlsanity.com/paper/2501.06286/) (2025)

## Abstract (arXiv)

Explicit decomposition modeling, which involves breaking down complex tasks into more straightforward and often more interpretable sub-tasks, has long been a central theme in developing robust and interpretable NLU systems. However, despite the many datasets and resources built as part of this effort, the majority have small-scale annotations and limited scope, which is insufficient to solve general decomposition tasks. In this paper, we look at large-scale intermediate pre-training of decomposition-based transformers using distant supervision from comparable texts, particularly large-scale parallel news. We show that with such intermediate pre-training, developing robust decomposition-based models for a diverse range of tasks becomes more feasible. For example, on semantic parsing, our model, DecompT5, improves 20% to 30% on two datasets, Overnight and TORQUE, over the baseline language model. We further use DecompT5 to build a novel decomposition-based QA system named DecompEntail, improving over state-of-the-art models, including GPT-3, on both HotpotQA and StrategyQA by 8% and 4%, respectively.