mlsanityData to 13 Oct 2026

Rethinking Search: Making Domain Experts out of Dilettantes

arXiv 2105.02274 ↗First posted 5 May 2021cs.IR cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

When experiencing an information need, users want to engage with a domain expert, but often turn to an information retrieval system, such as a search engine, instead. Classical information retrieval systems do not answer information needs directly, but instead provide references to (hopefully authoritative) answers. Successful question answering systems offer a limited corpus created on-demand by human experts, which is neither timely nor scalable. Pre-trained language models, by contrast, are capable of directly generating prose that may be responsive to an information need, but at present they are dilettantes rather than domain experts -- they do not have a true understanding of the world, they are prone to hallucinating, and crucially they are incapable of justifying their utterances by referring to supporting documents in the corpus they were trained over. This paper examines how ideas from classical information retrieval and pre-trained language models can be synthesized and evolved into systems that truly deliver on the promise of domain expert advice.

Key facts. Cited by 18 papers in the map (data through 13 Oct 2026). Builds on 68 of its 110 references that are in the map. First posted on arXiv 5 May 2021.

Builds on

68 of 110 references.

References · 68

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2020Language Models are Few-Shot Learnerscited by 32,320
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,549
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
  7. 2014Explaining and Harnessing Adversarial Examplescited by 7,336
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,884
  9. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,360
  10. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,133
  11. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,797
  12. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,467
38 more
  1. 2020Longformer: The Long-Document Transformercited by 3,393
  2. 2017Axiomatic Attribution for Deep Networkscited by 3,368
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,044
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,925
  5. 2018Deep contextualized word representationscited by 2,286
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,815
  7. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,322
  8. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,288
  9. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,169
  10. 2018A Survey Of Methods For Explaining Black Box Modelscited by 971
  11. 2015Pointer Networkscited by 884
  12. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 847
  13. 2020Efficient Transformers: A Surveycited by 785
  14. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 697
  15. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 693
  16. 2019How multilingual is Multilingual BERT?cited by 643
  17. 2017The NarrativeQA Reading Comprehension Challengecited by 592
  18. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  19. 2020Dataset Condensation with Gradient Matchingcited by 512
  20. 2020Towards a Human-like Open-Domain Chatbotcited by 469
  21. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  22. 2014Memory Networkscited by 360
  23. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  24. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 261
  25. 2018The Natural Language Decathlon: Multitask Learning as Question Answeringcited by 235
  26. 2016Key-Value Memory Networks for Directly Reading Documentscited by 223
  27. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 223
  28. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 222
  29. 2016A Decomposable Attention Model for Natural Language Inferencecited by 214
  30. 2016Ex Machina: Personal Attacks Seen at Scalecited by 209
  31. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  32. 2018Learning Latent Permutations with Gumbel-Sinkhorn Networkscited by 174
  33. 2018QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehensioncited by 164
  34. 2019Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answeringcited by 147
  35. 2017A Deep Relevance Matching Model for Ad-hoc Retrievalcited by 126
  36. 2017Simple and Effective Multi-Paragraph Reading Comprehensioncited by 120
  37. 2017Learned in Translation: Contextualized Word Vectorscited by 104
  38. 2017End-to-End Neural Ad-hoc Ranking with Kernel Poolingcited by 85

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 5; 2023: 4; 2024: 5; 2025: 2; 2026: 2.

18 papers.

Sort:

By area:

  1. 2024MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentationcited by 119
  2. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  3. 2022In conversation with Artificial Intelligence: aligning language models with human valuescited by 62
  4. 2022GERE: Generative Evidence Retrieval for Fact Verificationcited by 38
  5. 2022CorpusBrain: Pre-train a Generative Retrieval Model for Knowledge-Intensive Language Taskscited by 36
  6. 2024Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)cited by 20
  7. 2024CorpusLM: Towards a Unified Language Model on Corpus for Knowledge-Intensive Taskscited by 18
  8. 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspectivecited by 15
  9. 2023Context Aware Query Rewriting for Text Rankers using LLMcited by 14
  10. 2023ChatGPT Hallucinates when Attributing Answerscited by 14
  11. 2023Unifying Corroborative and Contributive Attributions in Large Language Modelscited by 10
  12. 2025Lightweight and Direct Document Relevance Optimization for Generative Information Retrievalcited by 9
6 more
  1. 2023Exploring the Practicality of Generative Retrieval on Dynamic Corporacited by 6
  2. 2025Advances in Artificial Intelligence: A Review for the Creative Industriescited by 2
  3. 2022The Role of Complex NLP in Transformers for Text Ranking?cited by 1
  4. 2026Analytical Searchcited by 1
  5. 2026A Parametric Memory Head for Continual Generative Retrievalcited by 1
  6. 2022Generative Long-form Question Answering: Relevance, Faithfulness and Succinctnesscited by 0