Analyzing and Characterizing User Intent in Information-seeking Conversations
Not on the map. Why?
Abstract (arXiv)
Understanding and characterizing how people interact in information-seeking conversations is crucial in developing conversational search systems. In this paper, we introduce a new dataset designed for this purpose and use it to analyze information-seeking conversations by user intent distribution, co-occurrence, and flow patterns. The MSDialog dataset is a labeled dialog dataset of question answering (QA) interactions between information seekers and providers from an online forum on Microsoft products. The dataset contains more than 2,000 multi-turn QA dialogs with 10,000 utterances that are annotated with user intent on the utterance level. Annotations were done using crowdsourcing. With MSDialog, we find some highly recurring patterns in user intent during an information-seeking process. They could be useful for designing conversational search systems. We will make our dataset freely available to encourage exploration of information-seeking conversation models.
Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Apr 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 6; 2022: 4; 2023: 6; 2024: 10; 2025: 6; 2026: 4.
36 papers.
Sort:
By area:
- 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
- 2024Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agentscited by 71
- 2023User Simulation for Evaluating Information Access Systemscited by 30
- 2021On the Calibration and Uncertainty of Neural Learning to Rank Modelscited by 24
- 2025Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wildcited by 24
- 2024A Survey on Recent Advances in Conversational Data Generationcited by 23
- 2024A Survey of Conversational Searchcited by 22
- 2022User Satisfaction Estimation with Sequential Dialogue Act Modeling in Goal-oriented Conversational Systemscited by 20
- 2024Self-seeding and Multi-intent Self-instructing LLMs for Generating Intent-aware Information-Seeking dialogscited by 18
- 2024STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agentscited by 17
- 2024Dial-In LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialoguescited by 11
- 2023Investigating Conversational Search Behavior For Domain Explorationcited by 9
24 more
- 2025Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agentscited by 8
- 2021We've had this conversation before: A Novel Approach to Measuring Dialog Similaritycited by 7
- 2022Mediators: Conversational Agents Explaining NLP Model Behaviorcited by 7
- 2024Learning to Ask: Conversational Product Search via Representation Learningcited by 7
- 2021Meta-evaluation of Conversational Search Evaluation Metricscited by 6
- 2024Engineering Conversational Search Systems: A Review of Applications, Architectures, and Functional Componentscited by 3
- 2025Trapped by Expectations: Functional Fixedness in LLM-Enabled Chat Searchcited by 3
- 2021POSSCORE: A Simple Yet Effective Evaluation of Conversational Search with Part of Speech Labellingcited by 2
- 2022Intent Recognition in Conversational Recommender Systemscited by 2
- 2023Efficient Uncertainty Estimation with Gaussian Process for Reliable Dialog Response Retrievalcited by 2
- 2024Corpus-informed Retrieval Augmented Generation of Clarifying Questionscited by 2
- 2024From Intents to Conversations: Generating Intent-Driven Dialogues with Contrastive Learning for Multi-Turn Classificationcited by 2
- 2023FCC: Fusing Conversation History and Candidate Provenance for Contextual Response Ranking in Dialogue Systemscited by 1
- 2023Intent-calibrated Self-training for Answer Selection in Open-domain Dialoguescited by 1
- 2025PSCon: Product Search Through Conversationscited by 1
- 2025IP-Dialog: Evaluating Implicit Personalization in Dialogue Systems with Synthetic Datacited by 1
- 2021Evaluating Predictive Uncertainty under Distributional Shift on Dialogue Datasetcited by 0
- 2022Sparse and Dense Approaches for the Full-rank Retrieval of Responses for Dialoguescited by 0
- 2023On the Calibration and Uncertainty with P'olya-Gamma Augmentation for Dialog Retrieval Modelscited by 0
- 2025Finding Diamonds in Conversation Haystacks: A Benchmark for Conversational Data Retrievalcited by 0
- 2026Do Images Clarify? A Study on the Effect of Images on Clarifying Questions in Conversational Searchcited by 0
- 2026Structure-Aware RAG: Structured Retrieval Augmented Generation from Noisy Data for Conversational Agentscited by 0
- 2026CIR at iKAT SCAI 2026: Exploring Clarification Need Prediction in Agentic Conversational Searchcited by 0
- 2026RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agentscited by 0
- 2026RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents17 Sep 2026
- 2026CIR at iKAT SCAI 2026: Exploring Clarification Need Prediction in Agentic Conversational Search22 Jul 2026
- 2026Structure-Aware RAG: Structured Retrieval Augmented Generation from Noisy Data for Conversational Agents23 May 2026
- 2026Do Images Clarify? A Study on the Effect of Images on Clarifying Questions in Conversational Search9 Feb 2026
- 2025Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents6 Oct 2025
- 2025Finding Diamonds in Conversation Haystacks: A Benchmark for Conversational Data Retrieval3 Oct 2025
- 2025IP-Dialog: Evaluating Implicit Personalization in Dialogue Systems with Synthetic Data3 Jun 2025
- 2025Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild21 May 2025
- 2025Trapped by Expectations: Functional Fixedness in LLM-Enabled Chat Search2 Apr 2025
- 2025PSCon: Product Search Through Conversations19 Feb 2025
- 2024Dial-In LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialogues12 Dec 2024
- 2024From Intents to Conversations: Generating Intent-Driven Dialogues with Contrastive Learning for Multi-Turn Classification21 Nov 2024
24 more
- 2024Learning to Ask: Conversational Product Search via Representation Learning18 Nov 2024
- 2024A Survey of Conversational Search21 Oct 2024
- 2024Corpus-informed Retrieval Augmented Generation of Clarifying Questions27 Sep 2024
- 2024Engineering Conversational Search Systems: A Review of Applications, Architectures, and Functional Components1 Jul 2024
- 2024STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agents20 May 2024
- 2024A Survey on Recent Advances in Conversational Data Generation12 May 2024
- 2024Self-seeding and Multi-intent Self-instructing LLMs for Generating Intent-aware Information-Seeking dialogs18 Feb 2024
- 2024Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents14 Feb 2024
- 2023Intent-calibrated Self-training for Answer Selection in Open-domain Dialogues13 Jul 2023
- 2023User Simulation for Evaluating Information Access Systems14 Jun 2023
- 2023FCC: Fusing Conversation History and Candidate Provenance for Contextual Response Ranking in Dialogue Systems31 Mar 2023
- 2023On the Calibration and Uncertainty with P'olya-Gamma Augmentation for Dialog Retrieval Models15 Mar 2023
- 2023Efficient Uncertainty Estimation with Gaussian Process for Reliable Dialog Response Retrieval15 Mar 2023
- 2023Investigating Conversational Search Behavior For Domain Exploration10 Jan 2023
- 2022Intent Recognition in Conversational Recommender Systems6 Dec 2022
- 2022Mediators: Conversational Agents Explaining NLP Model Behavior13 Jun 2022
- 2022Sparse and Dense Approaches for the Full-rank Retrieval of Responses for Dialogues22 Apr 2022
- 2022User Satisfaction Estimation with Sequential Dialogue Act Modeling in Goal-oriented Conversational Systems7 Feb 2022
- 2021We've had this conversation before: A Novel Approach to Measuring Dialog Similarity12 Oct 2021
- 2021POSSCORE: A Simple Yet Effective Evaluation of Conversational Search with Part of Speech Labelling7 Sep 2021
- 2021Evaluating Predictive Uncertainty under Distributional Shift on Dialogue Dataset1 Sep 2021
- 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension27 Jul 2021
- 2021Meta-evaluation of Conversational Search Evaluation Metrics27 Apr 2021
- 2021On the Calibration and Uncertainty of Neural Learning to Rank Models12 Jan 2021
- 2024Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agentscited by 71
- 2024STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agentscited by 17
- 2026CIR at iKAT SCAI 2026: Exploring Clarification Need Prediction in Agentic Conversational Searchcited by 0
- 2026CIR at iKAT SCAI 2026: Exploring Clarification Need Prediction in Agentic Conversational Search22 Jul 2026
- 2024STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agents20 May 2024
- 2024Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents14 Feb 2024
- 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
- 2023User Simulation for Evaluating Information Access Systemscited by 30
- 2021On the Calibration and Uncertainty of Neural Learning to Rank Modelscited by 24
- 2025Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wildcited by 24
- 2024A Survey on Recent Advances in Conversational Data Generationcited by 23
- 2024A Survey of Conversational Searchcited by 22
- 2022User Satisfaction Estimation with Sequential Dialogue Act Modeling in Goal-oriented Conversational Systemscited by 20
- 2024Dial-In LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialoguescited by 11
- 2023Investigating Conversational Search Behavior For Domain Explorationcited by 9
- 2025Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agentscited by 8
- 2021We've had this conversation before: A Novel Approach to Measuring Dialog Similaritycited by 7
- 2022Mediators: Conversational Agents Explaining NLP Model Behaviorcited by 7
12 most cited of 29 All 29 (JSON)
- 2026Do Images Clarify? A Study on the Effect of Images on Clarifying Questions in Conversational Search9 Feb 2026
- 2025Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents6 Oct 2025
- 2025Finding Diamonds in Conversation Haystacks: A Benchmark for Conversational Data Retrieval3 Oct 2025
- 2025IP-Dialog: Evaluating Implicit Personalization in Dialogue Systems with Synthetic Data3 Jun 2025
- 2025Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild21 May 2025
- 2025Trapped by Expectations: Functional Fixedness in LLM-Enabled Chat Search2 Apr 2025
- 2025PSCon: Product Search Through Conversations19 Feb 2025
- 2024Dial-In LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialogues12 Dec 2024
- 2024From Intents to Conversations: Generating Intent-Driven Dialogues with Contrastive Learning for Multi-Turn Classification21 Nov 2024
- 2024Learning to Ask: Conversational Product Search via Representation Learning18 Nov 2024
- 2024A Survey of Conversational Search21 Oct 2024
- 2024Engineering Conversational Search Systems: A Review of Applications, Architectures, and Functional Components1 Jul 2024
12 newest of 29 All 29 (JSON)