# Guided Transformer: Leveraging Multiple External Sources for Representation Learning in Conversational Search

arXiv: https://arxiv.org/abs/2006.07548 · HTML: https://mlsanity.com/paper/2006.07548/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2020.

## Built on by (most cited first, 23 of 23)

- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [ConvGQR: Generative Query Reformulation for Conversational Search](https://mlsanity.com/paper/2305.15645/) (2023)
- [Retrieval-Enhanced Machine Learning](https://mlsanity.com/paper/2205.01230/) (2022)
- [Evaluating Mixed-initiative Conversational Search Systems via User Simulation](https://mlsanity.com/paper/2204.08046/) (2022)
- [A Survey of Conversational Search](https://mlsanity.com/paper/2410.15576/) (2024)
- [A Three-Stage Learning Framework for Low-Resource Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2109.04096/) (2021)
- [You can't pick your neighbors, or can you? When and how to rely on retrieval in the kNN-LM](https://mlsanity.com/paper/2210.15859/) (2022)
- [Rethinking Privacy in Machine Learning Pipelines from an Information Flow Control Perspective](https://mlsanity.com/paper/2311.15792/) (2023)
- [Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots](https://mlsanity.com/paper/2103.09534/) (2021)
- [Analysing Mixed Initiatives and Search Strategies during Conversational Search](https://mlsanity.com/paper/2109.05955/) (2021)
- [UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost](https://mlsanity.com/paper/2104.04946/) (2021)
- [Retrieval-Enhanced Machine Learning: Synthesis and Opportunities](https://mlsanity.com/paper/2407.12982/) (2024)
- [Asking Multimodal Clarifying Questions in Mixed-Initiative Conversational Search](https://mlsanity.com/paper/2402.07742/) (2024)
- [A Graph-guided Multi-round Retrieval Method for Conversational Open-domain Question Answering](https://mlsanity.com/paper/2104.08443/) (2021)
- [Learning to Ask: Conversational Product Search via Representation Learning](https://mlsanity.com/paper/2411.14466/) (2024)
- [Knowledge-Enhanced Conversational Recommendation via Transformer-based Sequential Modelling](https://mlsanity.com/paper/2412.02415/) (2024)
- [Understanding User Intent Modeling for Conversational Recommender Systems: A Systematic Literature Review](https://mlsanity.com/paper/2308.08496/) (2023)
- [PGT: Pseudo Relevance Feedback Using a Graph-Based Transformer](https://mlsanity.com/paper/2101.07918/) (2021)
- [Keyword Extraction for Improved Document Retrieval in Conversational Search](https://mlsanity.com/paper/2109.05979/) (2021)
- [Towards Building Economic Models of Conversational Search](https://mlsanity.com/paper/2201.08742/) (2022)
- [The Contemporary Art of Image Search: Iterative User Intent Expansion via Vision-Language Model](https://mlsanity.com/paper/2312.01656/) (2023)
- [Do Images Clarify? A Study on the Effect of Images on Clarifying Questions in Conversational Search](https://mlsanity.com/paper/2602.08700/) (2026)
- [AIGQ: An End-to-End Hybrid Generative Architecture for E-commerce Query Recommendation](https://mlsanity.com/paper/2603.19710/) (2026)

## Abstract (arXiv)

Asking clarifying questions in response to ambiguous or faceted queries has been recognized as a useful technique for various information retrieval systems, especially conversational search systems with limited bandwidth interfaces. Analyzing and generating clarifying questions have been studied recently but the accurate utilization of user responses to clarifying questions has been relatively less explored. In this paper, we enrich the representations learned by Transformer networks using a novel attention mechanism from external information sources that weights each term in the conversation. We evaluate this Guided Transformer model in a conversational search scenario that includes clarifying questions. In our experiments, we use two separate external sources, including the top retrieved documents and a set of different possible clarifying questions for the query. We implement the proposed representation learning model for two downstream tasks in conversational search; document retrieval and next clarifying question selection. Our experiments use a public dataset for search clarification and demonstrate significant improvements compared to competitive baselines.