mlsanityData to 14 Oct 2026

Boosting Search Engines with Interactive Agents

arXiv 2109.00527 ↗First posted 1 Sep 2021cs.CL cs.AI cs.IRcited by 19

Not on the map. Why?

Abstract (arXiv)

This paper presents first successful steps in designing search agents that learn meta-strategies for iterative query refinement in information-seeking tasks. Our approach uses machine reading to guide the selection of refinement terms from aggregated search results. Agents are then empowered with simple but effective search operators to exert fine-grained and transparent control over queries and search results. We develop a novel way of generating synthetic search sessions, which leverages the power of transformer-based language models through (self-)supervised learning. We also present a reinforcement learning agent with dynamically constrained actions that learns interactive search strategies from scratch. Our search agents obtain retrieval and answer quality performance comparable to recent neural methods, using only a traditional term-based BM25 ranking function and interpretable discrete reranking and filtering actions.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 38 of its 82 references that are in the map. First posted on arXiv 1 Sep 2021.

Builds on

38 of 82 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  9. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  10. 2016Generative Adversarial Imitation Learningcited by 1,597
  11. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  12. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
26 more
  1. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  2. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  3. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  4. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  5. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  6. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  7. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  8. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  9. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  10. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 224
  11. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
  12. 2021Overview of the TREC 2020 deep learning trackcited by 196
  13. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 162
  14. 2020Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasetscited by 138
  15. 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 135
  16. 2013A Theoretical Analysis of NDCG Type Ranking Measurescited by 119
  17. 2019On the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 82
  18. 2019Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answeringcited by 48
  19. 2017Ask the Right Questions: Active Question Reformulation with Reinforcement Learningcited by 44
  20. 2017Task-Oriented Query Reformulation with Reinforcement Learningcited by 39
  21. 2021Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 33
  22. 2019Hierarchical Decision Making by Generating and Following Natural Language Instructionscited by 30
  23. 2016Improving Information Extraction by Acquiring External Evidence with Reinforcement Learningcited by 16
  24. 2019Interactive Machine Comprehension with Information Seeking Agentscited by 13
  25. 2017Grounding Language for Transfer in Deep Reinforcement Learningcited by 5
  26. 2013Query Expansion Using Term Distribution and Term Associationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 4; 2024: 4; 2025: 0; 2026: 2.

19 papers.

Sort:

By area:

  1. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  2. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  3. 2023Augmented Language Models: a Surveycited by 403
  4. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353
  5. 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 160
  6. 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
  7. 2024Bridging the Preference Gap between Retrievers and LLMscited by 69
  8. 2021CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learningcited by 68
  9. 2021Reason first, then respond: Modular Generation for Knowledge-infused Dialoguecited by 32
  10. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processingcited by 25
  11. 2022Visconde: Multi-document QA with GPT-3 and Neural Rerankingcited by 23
  12. 20231-PAGER: One Pass Answer Generation and Evidence Retrievalcited by 7
7 more
  1. 2022Zero-Shot Retrieval with Search Agents and Hybrid Environmentscited by 6
  2. 2022Decoding a Neural Retriever's Latent Space for Query Suggestioncited by 5
  3. 2024From RAG to RICHES: Retrieval Interlaced with Sequence Generationcited by 5
  4. 2024An Extremely Data-efficient and Generative LLM-based Reinforcement Learning Agent for Recommenderscited by 2
  5. 2022Cheater's Bowl: Human vs. Computer Search Strategies for Open-Domain Question Answeringcited by 0
  6. 2026DeLIVeR: Decomposed Learning for Information-grounded Veracity Recognition via Reinforced Knowledge Graph Explorationcited by 0
  7. 2026EMBL AI Librarian: Life-Sciences Knowledge Layer for AI Agentscited by 0