# LLM Benchmark-User Need Misalignment for Climate Change

arXiv: https://arxiv.org/abs/2603.26106 · HTML: https://mlsanity.com/paper/2603.26106/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 59 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Builds on (most cited first, 13 of 13)

- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset](https://mlsanity.com/paper/2309.11998/) (2023)
- [CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims](https://mlsanity.com/paper/2012.00614/) (2020)
- [On the Risk of Misinformation Pollution with Large Language Models](https://mlsanity.com/paper/2305.13661/) (2023)
- [The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions](https://mlsanity.com/paper/2310.12418/) (2023)
- [Using Large Language Models to Generate, Validate, and Apply User Intent Taxonomies](https://mlsanity.com/paper/2309.13063/) (2023)
- [Assessing Large Language Models on Climate Information](https://mlsanity.com/paper/2310.02932/) (2023)
- [ClimaQA: An Automated Evaluation Framework for Climate Question Answering Models](https://mlsanity.com/paper/2410.16701/) (2024)
- [ClimRetrieve: A Benchmarking Dataset for Information Retrieval from Corporate Climate Disclosures](https://mlsanity.com/paper/2406.09818/) (2024)
- [HICode: Hierarchical Inductive Coding with LLMs](https://mlsanity.com/paper/2509.17946/) (2025)
- [Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM](https://mlsanity.com/paper/2312.09366/) (2023)

## Abstract (arXiv)

Climate change is a major socio-scientific issue shapes public decision-making and policy discussions. As large language models (LLMs) increasingly serve as an interface for accessing climate knowledge, whether existing benchmarks reflect user needs is critical for evaluating LLM in real-world settings. We propose a Proactive Knowledge Behaviors Framework that captures the different human-human and human-AI knowledge seeking and provision behaviors. We further develop a Topic-Intent-Form taxonomy and apply it to analyze climate-related data representing different knowledge behaviors. Our results reveal a substantial mismatch between current benchmarks and real-world user needs, while knowledge interaction patterns between humans and LLMs closely resemble those in human-human interactions. These findings provide actionable guidance for benchmark design, RAG system development, and LLM training. Code is available at https://github.com/OuchengLiu/LLM-Misalign-Climate-Change.