# MultiClimate: Multimodal Stance Detection on Climate Change Videos

arXiv: https://arxiv.org/abs/2409.18346 · HTML: https://mlsanity.com/paper/2409.18346/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://mlsanity.com/paper/2103.00242/) (2021)
- [Will-They-Won't-They: A Very Large Dataset for Stance Detection on Twitter](https://mlsanity.com/paper/2005.00388/) (2020)
- [Stance Detection with Collaborative Role-Infused LLM-Based Agents](https://mlsanity.com/paper/2310.10467/) (2023)
- [Multi-modal Stance Detection: New Datasets and Model](https://mlsanity.com/paper/2402.14298/) (2024)

## Built on by (most cited first, 7 of 7)

- [Exploring Vision Language Models for Multimodal and Multilingual Stance Detection](https://mlsanity.com/paper/2501.17654/) (2025)
- [CliME: Evaluating Multimodal Climate Discourse on Social Media and the Climate Alignment Quotient (CAQ)](https://mlsanity.com/paper/2504.03906/) (2025)
- [Two Stage Context Learning with Large Language Models for Multimodal Stance Detection on Climate Change](https://mlsanity.com/paper/2509.08024/) (2025)
- [A Multimodal Benchmark for Framing of Oil & Gas Advertising and Potential Greenwashing Detection](https://mlsanity.com/paper/2510.21679/) (2025)
- [TikStance: A Multimodal and Hierarchical Dataset for Multi-target Stance Analysis in TikTok Political Conversations](https://mlsanity.com/paper/2607.15240/) (2026)
- [StanceFlip: A Comprehensive Multi-Dimensional Benchmark for Multimodal Conversational Stance Flipping Forecasting](https://mlsanity.com/paper/2607.24191/) (2026)
- [MMDS-Bench: Benchmarking Multimodal Large Language Models on Dynamic Stance in Social Media Interactions](https://mlsanity.com/paper/2608.30903/) (2026)

## Abstract (arXiv)

Climate change (CC) has attracted increasing attention in NLP in recent years. However, detecting the stance on CC in multimodal data is understudied and remains challenging due to a lack of reliable datasets. To improve the understanding of public opinions and communication strategies, this paper presents MultiClimate, the first open-source manually-annotated stance detection dataset with 100 CC-related YouTube videos and 4,209 frame-transcript pairs. We deploy state-of-the-art vision and language models, as well as multimodal models for MultiClimate stance detection. Results show that text-only BERT significantly outperforms image-only ResNet50 and ViT. Combining both modalities achieves state-of-the-art, 0.747/0.749 in accuracy/F1. Our 100M-sized fusion models also beat CLIP and BLIP, as well as the much larger 9B-sized multimodal IDEFICS and text-only Llama3 and Gemma2, indicating that multimodal stance detection remains challenging for large language models. Our code, dataset, as well as supplementary materials, are available at https://github.com/werywjw/MultiClimate.