# On the Reliability of Large Language Models to Misinformed and Demographically-Informed Prompts

arXiv: https://arxiv.org/abs/2410.10850 · HTML: https://mlsanity.com/paper/2410.10850/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 6 Oct 2024.

## Builds on (most cited first, 22 of 22)

- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge](https://mlsanity.com/paper/2303.14070/) (2023)
- [Multimodal Foundation Models: From Specialists to General-Purpose Assistants](https://mlsanity.com/paper/2309.10020/) (2023)
- [CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims](https://mlsanity.com/paper/2012.00614/) (2020)
- [GPT detectors are biased against non-native English writers](https://mlsanity.com/paper/2304.02819/) (2023)
- [Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices](https://mlsanity.com/paper/1906.09208/) (2019)
- [MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare](https://mlsanity.com/paper/2110.15621/) (2021)
- [ClimateBert: A Pretrained Language Model for Climate-Related Text](https://mlsanity.com/paper/2110.12010/) (2021)
- [DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultation](https://mlsanity.com/paper/2308.14346/) (2023)
- [Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing](https://mlsanity.com/paper/2001.00964/) (2020)
- [Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data](https://mlsanity.com/paper/2307.14385/) (2023)
- [Design Principles for Generative AI Applications](https://mlsanity.com/paper/2401.14484/) (2024)
- [Are Large Language Models Ready for Healthcare? A Comparative Study on Clinical Language Understanding](https://mlsanity.com/paper/2304.05368/) (2023)
- [CHATREPORT: Democratizing Sustainability Disclosure Analysis through LLM-based Tools](https://mlsanity.com/paper/2307.15770/) (2023)
- [Exciting, Useful, Worrying, Futuristic: Public Perception of Artificial Intelligence in 8 Countries](https://mlsanity.com/paper/2001.00081/) (2019)
- [Enhancing Large Language Models with Climate Resources](https://mlsanity.com/paper/2304.00116/) (2023)
- [Aligning Large Language Models for Clinical Tasks](https://mlsanity.com/paper/2309.02884/) (2023)
- [CliMedBERT: A Pre-trained Language Model for Climate and Health-related Text](https://mlsanity.com/paper/2212.00689/) (2022)
- [Fine-tuning ClimateBert transformer with ClimaText for the disclosure analysis of climate-related financial risks](https://mlsanity.com/paper/2303.13373/) (2023)

## Built on by (most cited first, 2 of 2)

- [Mitigating Watermark Forgery in Generative Models via Randomized Key Selection](https://mlsanity.com/paper/2507.07871/) (2025)
- [Context Discipline and Performance Correlation: Analyzing LLM Performance and Quality Degradation Under Varying Context Lengths](https://mlsanity.com/paper/2601.11564/) (2025)

## Abstract (arXiv)

We investigate and observe the behaviour and performance of Large Language Model (LLM)-backed chatbots in addressing misinformed prompts and questions with demographic information within the domains of Climate Change and Mental Health. Through a combination of quantitative and qualitative methods, we assess the chatbots' ability to discern the veracity of statements, their adherence to facts, and the presence of bias or misinformation in their responses. Our quantitative analysis using True/False questions reveals that these chatbots can be relied on to give the right answers to these close-ended questions. However, the qualitative insights, gathered from domain experts, shows that there are still concerns regarding privacy, ethical implications, and the necessity for chatbots to direct users to professional services. We conclude that while these chatbots hold significant promise, their deployment in sensitive areas necessitates careful consideration, ethical oversight, and rigorous refinement to ensure they serve as a beneficial augmentation to human expertise rather than an autonomous solution.