# Climate-Eval: A Comprehensive Benchmark for NLP Tasks Related to Climate Change

arXiv: https://arxiv.org/abs/2505.18653 · HTML: https://mlsanity.com/paper/2505.18653/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 24 May 2025.

## Builds on (most cited first, 9 of 9)

- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018)
- [CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims](https://mlsanity.com/paper/2012.00614/) (2020)
- [The Semantic Scholar Open Data Platform](https://mlsanity.com/paper/2301.10140/) (2023)
- [ClimaText: A Dataset for Climate Change Topic Detection](https://mlsanity.com/paper/2012.00483/) (2020)
- [ClimateBERT-NetZero: Detecting and Assessing Net Zero and Reduction Targets](https://mlsanity.com/paper/2310.08096/) (2023)
- [A Multi-task Model for Sentiment Aided Stance Detection of Climate Change Tweets](https://mlsanity.com/paper/2211.03533/) (2022)
- [Towards Answering Climate Questionnaires from Unstructured Climate Reports](https://mlsanity.com/paper/2301.04253/) (2023)
- [INDUS: Effective and Efficient Language Models for Scientific Applications](https://mlsanity.com/paper/2405.10725/) (2024)

## Built on by (most cited first, 1 of 1)

- [How Large Language Models Systematically Misrepresent American Climate Opinions](https://mlsanity.com/paper/2512.23889/) (2025)

## Abstract (arXiv)

Climate-Eval is a comprehensive benchmark designed to evaluate natural language processing models across a broad range of tasks related to climate change. Climate-Eval aggregates existing datasets along with a newly developed news classification dataset, created specifically for this release. This results in a benchmark of 25 tasks based on 13 datasets, covering key aspects of climate discourse, including text classification, question answering, and information extraction. Our benchmark provides a standardized evaluation suite for systematically assessing the performance of large language models (LLMs) on these tasks. Additionally, we conduct an extensive evaluation of open-source LLMs (ranging from 2B to 70B parameters) in both zero-shot and few-shot settings, analyzing their strengths and limitations in the domain of climate change.