mlsanityData to 14 Oct 2026

Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Texts

arXiv 2210.16865 ↗First posted 30 Oct 2022cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

Explicit decomposition modeling, which involves breaking down complex tasks into more straightforward and often more interpretable sub-tasks, has long been a central theme in developing robust and interpretable NLU systems. However, despite the many datasets and resources built as part of this effort, the majority have small-scale annotations and limited scope, which is insufficient to solve general decomposition tasks. In this paper, we look at large-scale intermediate pre-training of decomposition-based transformers using distant supervision from comparable texts, particularly large-scale parallel news. We show that with such intermediate pre-training, developing robust decomposition-based models for a diverse range of tasks becomes more feasible. For example, on semantic parsing, our model, DecompT5, improves 20% to 30% on two datasets, Overnight and TORQUE, over the baseline language model. We further use DecompT5 to build a novel decomposition-based QA system named DecompEntail, improving over state-of-the-art models, including GPT-3, on both HotpotQA and StrategyQA by 8% and 4%, respectively.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 31 of its 39 references that are in the map. First posted on arXiv 30 Oct 2022.

Builds on

31 of 39 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  8. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  9. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  10. 2018Annotation Artifacts in Natural Language Inference Datacited by 671
  11. 2019Defending Against Neural Fake Newscited by 559
  12. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 476
19 more
  1. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  2. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 410
  3. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 402
  4. 2015Neural Module Networkscited by 376
  5. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 301
  6. 2021Constrained Language Models Yield Few-Shot Semantic Parserscited by 163
  7. 2020Break It Down: A Question Understanding Benchmarkcited by 131
  8. 2019"Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understandingcited by 123
  9. 2019Multi-hop Reading Comprehension through Question Decomposition and Rescoringcited by 119
  10. 2020Unsupervised Question Decomposition for Question Answeringcited by 118
  11. 2019Neural Module Networks for Reasoning over Textcited by 66
  12. 2020TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questionscited by 65
  13. 2020Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Modelscited by 65
  14. 2020Temporal Reasoning on Implicit Events from Distant Supervisioncited by 58
  15. 2020Temporal Common Sense Acquisition with Minimal Supervisioncited by 39
  16. 2021Challenges in Generalization in Open Domain Question Answeringcited by 30
  17. 2020Incidental Supervision: Moving beyond Supervised Learningcited by 13
  18. 2021Hey AI, Can You Solve Complex Tasks by Talking to Agents?cited by 13
  19. 2020Effective Distant Supervision for Temporal Relation Extractioncited by 8

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 6; 2024: 6; 2025: 5.

18 papers.

Sort:

By area:

  1. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 263
  2. 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasonerscited by 73
  3. 2023Natural Language Reasoning, A Surveycited by 61
  4. 2023Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representationscited by 20
  5. 2022Generic Temporal Reasoning with Differential Analysis and Explanationcited by 18
  6. 2024BIRD: A Trustworthy Bayesian Inference Framework for Large Language Modelscited by 18
  7. 2023Iterated Decomposition: Improving Science Q&A by Supervising Reasoning Processescited by 15
  8. 2023DORIS-MAE: Scientific Document Retrieval using Multi-level Aspect-based Queriescited by 11
  9. 2024Large Language Models are Contrastive Reasonerscited by 11
  10. 2024Retrieval-Enhanced Machine Learning: Synthesis and Opportunitiescited by 11
  11. 2025Question Decomposition for Retrieval-Augmented Generationcited by 11
  12. 2023DialCoT Meets PPO: Decomposing and Exploring Reasoning Paths in Smaller Language Modelscited by 6
6 more
  1. 2025POQD: Performance-Oriented Query Decomposer for Multi-vector retrievalcited by 5
  2. 2024DEXTER: A Benchmark for open-domain Complex Question Answering using LLMscited by 4
  3. 2025RECAP: Transparent Inference-Time Emotion Alignment for Medical Dialogue Systemscited by 3
  4. 2025Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainabilitycited by 2
  5. 2023Building Interpretable and Reliable Open Information Retriever for New Domains Overnightcited by 1
  6. 2025Bactrainus: Optimizing Large Language Models for Multi-hop Complex Question Answering Taskscited by 1