# ClimateChat: Designing Data and Methods for Instruction Tuning LLMs to Answer Climate Change Queries

arXiv: https://arxiv.org/abs/2506.13796 · HTML: https://mlsanity.com/paper/2506.13796/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?](https://mlsanity.com/paper/2405.05904/) (2024)
- [#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models](https://mlsanity.com/paper/2308.07074/) (2023)
- [ClimateBert: A Pretrained Language Model for Climate-Related Text](https://mlsanity.com/paper/2110.12010/) (2021)
- [Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases](https://mlsanity.com/paper/2303.14742/) (2023)
- [K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization](https://mlsanity.com/paper/2306.05064/) (2023)
- [ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Change](https://mlsanity.com/paper/2401.09646/) (2024)
- [chatClimate: Grounding Conversational AI in Climate Science](https://mlsanity.com/paper/2304.05510/) (2023)
- [Environmental Claim Detection](https://mlsanity.com/paper/2209.00507/) (2022)
- [Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM](https://mlsanity.com/paper/2312.09366/) (2023)
- [Climate change sentiment on Twitter: An unsolicited public opinion poll](https://mlsanity.com/paper/1505.03804/) (2015)
- [Analyzing Regional Impacts of Climate Change using Natural Language Processing Techniques](https://mlsanity.com/paper/2401.06817/) (2024)

## Built on by (most cited first, 3 of 3)

- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [TagRouter: Learning Route to LLMs through Tags for Open-Domain Text Generation Tasks](https://mlsanity.com/paper/2506.12473/) (2025)
- [Earth Science Foundation Models: From Perception to Reasoning and Discovery](https://mlsanity.com/paper/2605.12542/) (2026)

## Abstract (arXiv)

As the issue of global climate change becomes increasingly severe, the demand for research in climate science continues to grow. Natural language processing technologies, represented by Large Language Models (LLMs), have been widely applied to climate change-specific research, providing essential information support for decision-makers and the public. Some studies have improved model performance on relevant tasks by constructing climate change-related instruction data and instruction-tuning LLMs. However, current research remains inadequate in efficiently producing large volumes of high-precision instruction data for climate change, which limits further development of climate change LLMs. This study introduces an automated method for constructing instruction data. The method generates instructions using facts and background knowledge from documents and enhances the diversity of the instruction data through web scraping and the collection of seed instructions. Using this method, we constructed a climate change instruction dataset, named ClimateChat-Corpus, which was used to fine-tune open-source LLMs, resulting in an LLM named ClimateChat. Evaluation results show that ClimateChat significantly improves performance on climate change question-and-answer tasks. Additionally, we evaluated the impact of different base models and instruction data on LLM performance and demonstrated its capability to adapt to a wide range of climate change scientific discovery tasks, emphasizing the importance of selecting an appropriate base model for instruction tuning. This research provides valuable references and empirical support for constructing climate change instruction data and training climate change-specific LLMs.