# Powering an AI Chatbot with Expert Sourcing to Support Credible Health Information Access

arXiv: https://arxiv.org/abs/2301.10710 · HTML: https://mlsanity.com/paper/2301.10710/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jan 2023.

## Built on by (most cited first, 9 of 9)

- [How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensions](https://mlsanity.com/paper/2406.14805/) (2024)
- [ComPeer: A Generative Conversational Agent for Proactive Peer Support](https://mlsanity.com/paper/2407.18064/) (2024)
- [Generative Echo Chamber? Effects of LLM-Powered Search Systems on Diverse Information Seeking](https://mlsanity.com/paper/2402.05880/) (2024)
- [Can LLMs be Good Financial Advisors?: An Initial Study in Personal Decision Making for Optimized Outcomes](https://mlsanity.com/paper/2307.07422/) (2023)
- [CataractBot: An LLM-Powered Expert-in-the-Loop Chatbot for Cataract Patients](https://mlsanity.com/paper/2402.04620/) (2024)
- [Inform the uninformed: Improving Online Informed Consent Reading with an AI-Powered Chatbot](https://mlsanity.com/paper/2302.00832/) (2023)
- [HEALTH-PARIKSHA: Assessing RAG Models for Health Chatbots in Real-World Multilingual Settings](https://mlsanity.com/paper/2410.13671/) (2024)
- [Fine-Tuning LLMs for Reliable Medical Question-Answering Services](https://mlsanity.com/paper/2410.16088/) (2024)
- [Talk2Care: Facilitating Asynchronous Patient-Provider Communication with Large-Language-Model](https://mlsanity.com/paper/2309.09357/) (2023)

## Abstract (arXiv)

During a public health crisis like the COVID-19 pandemic, a credible and easy-to-access information portal is highly desirable. It helps with disease prevention, public health planning, and misinformation mitigation. However, creating such an information portal is challenging because 1) domain expertise is required to identify and curate credible and intelligible content, 2) the information needs to be updated promptly in response to the fast-changing environment, and 3) the information should be easily accessible by the general public; which is particularly difficult when most people do not have the domain expertise about the crisis. In this paper, we presented an expert-sourcing framework and created Jennifer, an AI chatbot, which serves as a credible and easy-to-access information portal for individuals during the COVID-19 pandemic. Jennifer was created by a team of over 150 scientists and health professionals around the world, deployed in the real world and answered thousands of user questions about COVID-19. We evaluated Jennifer from two key stakeholders' perspectives, expert volunteers and information seekers. We first interviewed experts who contributed to the collaborative creation of Jennifer to learn about the challenges in the process and opportunities for future improvement. We then conducted an online experiment that examined Jennifer's effectiveness in supporting information seekers in locating COVID-19 information and gaining their trust. We share the key lessons learned and discuss design implications for building expert-sourced and AI-powered information portals, along with the risks and opportunities of misinformation mitigation and beyond.