# Climate Change from Large Language Models

arXiv: https://arxiv.org/abs/2312.11985 · HTML: https://mlsanity.com/paper/2312.11985/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 19 Dec 2023.

## Builds on (most cited first, 20 of 20)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [Zephyr: Direct Distillation of LM Alignment](https://mlsanity.com/paper/2310.16944/) (2023)
- [Tackling Climate Change with Machine Learning](https://mlsanity.com/paper/1906.05433/) (2019)
- [The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink](https://mlsanity.com/paper/2204.05149/) (2022)
- [ClimateBert: A Pretrained Language Model for Climate-Related Text](https://mlsanity.com/paper/2110.12010/) (2021)
- [KoLA: Carefully Benchmarking World Knowledge of Large Language Models](https://mlsanity.com/paper/2306.09296/) (2023)
- [chatClimate: Grounding Conversational AI in Climate Science](https://mlsanity.com/paper/2304.05510/) (2023)
- [Enhancing Large Language Models with Climate Resources](https://mlsanity.com/paper/2304.00116/) (2023)
- [Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation](https://mlsanity.com/paper/2109.01048/) (2021)
- [MetaAID: A Flexible Framework for Developing Metaverse Applications via AI Technology and Human Editing](https://mlsanity.com/paper/2204.01614/) (2022)
- [Environmental Claim Detection](https://mlsanity.com/paper/2209.00507/) (2022)
- [Financial data analysis application via multi-strategy text processing](https://mlsanity.com/paper/2204.11394/) (2022)
- [MetaAID 2.0: An Extensible Framework for Developing Metaverse Applications via Human-controllable Pre-trained Models](https://mlsanity.com/paper/2302.13173/) (2023)
- [Industry Risk Assessment via Hierarchical Financial Data Using Stock Market Sentiment Indicators](https://mlsanity.com/paper/2303.02707/) (2023)
- [How GPT-3 responds to different publics on climate change and Black Lives Matter: A critical appraisal of equity in conversational AI](https://mlsanity.com/paper/2209.13627/) (2022)

## Built on by (most cited first, 6 of 6)

- [ClimaQA: An Automated Evaluation Framework for Climate Question Answering Models](https://mlsanity.com/paper/2410.16701/) (2024)
- [Exploring Large Language Models for Climate Forecasting](https://mlsanity.com/paper/2411.13724/) (2024)
- [Assessing Generative Language Models in Classification Tasks: Performance and Self-Evaluation Capabilities in the Environmental and Climate Change Domain](https://mlsanity.com/paper/2408.17362/) (2024)
- [SimulRAG: Simulator-based RAG for Grounding LLMs in Long-form Scientific QA](https://mlsanity.com/paper/2509.25459/) (2025)
- [ClimaEmpact: Domain-Aligned Small Language Models and Datasets for Extreme Weather Analytics](https://mlsanity.com/paper/2504.19066/) (2025)
- [MMClima: A Framework for Multimodal Climate Science Data and Evaluation](https://mlsanity.com/paper/2606.10194/) (2026)

## Abstract (arXiv)

Climate change poses grave challenges, demanding widespread understanding and low-carbon lifestyle awareness. Large language models (LLMs) offer a powerful tool to address this crisis, yet comprehensive evaluations of their climate-crisis knowledge are lacking. This paper proposes an automated evaluation framework to assess climate-crisis knowledge within LLMs. We adopt a hybrid approach for data acquisition, combining data synthesis and manual collection, to compile a diverse set of questions encompassing various aspects of climate change. Utilizing prompt engineering based on the compiled questions, we evaluate the model's knowledge by analyzing its generated answers. Furthermore, we introduce a comprehensive set of metrics to assess climate-crisis knowledge, encompassing indicators from 10 distinct perspectives. These metrics provide a multifaceted evaluation, enabling a nuanced understanding of the LLMs' climate crisis comprehension. The experimental results demonstrate the efficacy of our proposed method. In our evaluation utilizing diverse high-performing LLMs, we discovered that while LLMs possess considerable climate-related knowledge, there are shortcomings in terms of timeliness, indicating a need for continuous updating and refinement of their climate-related content.