mlsanityData to 14 Oct 2026

Chain-of-Skills: A Configurable Model for Open-domain Question Answering

arXiv 2305.03130 ↗First posted 4 May 2023cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

The retrieval model is an indispensable component for real-world knowledge-intensive tasks, e.g., open-domain question answering (ODQA). As separate retrieval skills are annotated for different datasets, recent work focuses on customized methods, limiting the model transferability and scalability. In this work, we propose a modular retriever where individual modules correspond to key skills that can be reused across datasets. Our approach supports flexible skill configurations based on the target domain to boost performance. To mitigate task interference, we design a novel modularization parameterization inspired by sparse Transformer. We demonstrate that our model can benefit from self-supervised pretraining on Wikipedia and fine-tuning using multiple ODQA datasets, both in a multi-task fashion. Our approach outperforms recent self-supervised retrievers in zero-shot evaluations and achieves state-of-the-art fine-tuned retrieval performance on NQ, HotpotQA and OTT-QA.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 56 of its 71 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

56 of 71 references.

References · 56

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2016Layer Normalizationcited by 5,669
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  9. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  10. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  11. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  12. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
38 more
  1. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  2. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  3. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  4. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  5. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  6. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 350
  7. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 264
  8. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  9. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  10. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
  11. 2020Open Question Answering over Tables and Textcited by 163
  12. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 162
  13. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  14. 2019Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answeringcited by 147
  15. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  16. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  17. 2021Taming Sparsely Activated Transformer with Stochastic Expertscited by 113
  18. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 109
  19. 2019Hierarchical Graph Network for Multi-hop Question Answeringcited by 83
  20. 2019Cognitive Graph for Multi-Hop Reading Comprehension at Scalecited by 65
  21. 2021Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrievalcited by 63
  22. 2019Answering Complex Open-domain Questions Through Iterative Query Generationcited by 60
  23. 2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 60
  24. 2021Domain-matched Pre-training Tasks for Dense Retrievalcited by 53
  25. 2019Revealing the Importance of Semantic Retrieval for Machine Reading at Scalecited by 50
  26. 2021Learning to Retrieve Passages without Supervisioncited by 46
  27. 2020Answering Open-Domain Questions of Varying Reasoning Steps from Textcited by 43
  28. 2021UnitedQA: A Hybrid Approach for Open Domain Question Answeringcited by 42
  29. 2021R2-D2: A Modular Baseline for Open-Domain Question Answeringcited by 37
  30. 2019Multi-Hop Paragraph Retrieval for Open-Domain Question Answeringcited by 34
  31. 2021Open Domain Question Answering with A Unified Knowledge Interfacecited by 32
  32. 2021Knowledge-Rich Self-Supervision for Biomedical Entity Linkingcited by 31
  33. 2021Adaptive Information Seeking for Open-Domain Question Answeringcited by 29
  34. 2022Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learningcited by 24
  35. 2020HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questionscited by 19
  36. 2021You Only Need One Model for Open-domain Question Answeringcited by 17
  37. 2022Mixed-modality Representation Learning and Pre-training for Joint Table-and-Text Retrieval in OpenQAcited by 17
  38. 2022Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answeringcited by 15

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 8; 2025: 2; 2026: 4.

19 papers.

Sort:

By area:

  1. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 184
  2. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 108
  3. 2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88
  4. 2024StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurizationcited by 71
  5. 2023LASER: LLM Agent with State-Space Exploration for Web Navigationcited by 66
  6. 2023Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representationscited by 20
  7. 2023Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understandingcited by 16
  8. 2024SPAGHETTI: Open-Domain Question Answering from Heterogeneous Data Sources with Retrieval and Semantic Parsingcited by 11
  9. 2024A Unified Taxonomy-Guided Instruction Tuning Framework for Entity Set Expansion and Taxonomy Expansioncited by 8
  10. 2024Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Modelscited by 7
  11. 2025Hybrid Graphs for Table-and-Text based Question Answering using LLMscited by 7
  12. 2024TTQA-RS- A break-down prompting approach for Multi-hop Table-Text Question Answering with Reasoning and Summarizationcited by 6
7 more
  1. 2026Retrieval from Within: An Intrinsic Capability of Attention-Based Modelscited by 2
  2. 2024Momentum Posterior Regularization for Multi-hop Dense Retrievalcited by 1
  3. 2026HELIOS: Harmonizing Early Fusion, Late Fusion, and LLM Reasoning for Multi-Granular Table-Text Retrievalcited by 1
  4. 2024Question Answering with Texts and Tables through Deep Reinforcement Learningcited by 0
  5. 2025SeBERTis: A Framework for Producing Classifiers of Security-Related Issue Reportscited by 0
  6. 2026SAGE: Structure Aware Graph Expansion for Retrieval of Heterogeneous Datacited by 0
  7. 2026SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tablescited by 0