YIELD: A Large-Scale Dataset and Evaluation Framework for Information Elicitation Agents
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Most conversational agents (CAs) are designed to satisfy user needs through user-driven interactions. However, many real-world settings, such as academic interviewing, judicial proceedings, and journalistic investigations, involve broader institutional decision-making processes and require agents that can elicit information from users. In this paper, we introduce Information Elicitation Agents (IEAs) in which the agent's goal is to elicit information from users to support the agent's institutional or task-oriented objectives. To enable systematic research on this setting, we present YIELD, a 26M-token dataset of 2,281 ethically sourced, human-to-human dialogues. Moreover, we formalize information elicitation as a finite-horizon POMDP and propose novel metrics tailored to IEAs. Pilot experiments on multiple foundation LLMs show that training on YIELD improves their alignment with real elicitation behavior and findings are corroborated by human evaluation. We release YIELD under CC BY 4.0. The dataset, project code, evaluation tools, and fine-tuned model adapters are available at: https://github.com/infosenselab/yield.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 9 of its 39 references that are in the map. First posted on arXiv 13 Apr 2026.
Builds on
9 of 39 references.
References · 9
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
- 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
- 2019Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetcited by 426
- 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 220
- 2019What makes a good conversation? How controllable attributes affect human judgmentscited by 140
- 2024Towards Human-centered Proactive Conversational Agentscited by 48
- 2024LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluationcited by 12
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate
- 2025InfoQuest: Evaluating Multi-Turn Dialogue Agents for Open-Ended Conversations with Hidden Context
- 2025MimiTalk: Revolutionizing Qualitative Research with Dual-Agent AI
- 2022Improving alignment of dialogue agents via targeted human judgements
- 2026Operationalising Multi-Dimensional Evaluation for Conversational Agents: A Scalable, Governed Pipeline with Selective Re-evaluation and Model Benchmarking
- 2026On Open-Ended Information Seeking for Information Elicitation Agents
- 2026Clarify User Expertise: Towards Proactive Conversational Agents Tailoring Responses to User Proficiency
- 2026Talk, Evaluate, Diagnose: User-aware Agent Evaluation with Automated Error Analysis