# TATA: Stance Detection via Topic-Agnostic and Topic-Aware Embeddings

arXiv: https://arxiv.org/abs/2310.14450 · HTML: https://mlsanity.com/paper/2310.14450/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 50 references that are in the map.
- First posted on arXiv 22 Oct 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations](https://mlsanity.com/paper/2010.03640/) (2020)
- [BertGCN: Transductive Text Classification by Combining GCN and BERT](https://mlsanity.com/paper/2105.05727/) (2021)
- [ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding](https://mlsanity.com/paper/2109.04380/) (2021)
- [Improved Embeddings with Easy Positive Triplet Mining](https://mlsanity.com/paper/1904.04370/) (2019)
- [Contrastive Learning for Image Captioning](https://mlsanity.com/paper/1710.02534/) (2017)
- [Unsupervised User Stance Detection on Twitter](https://mlsanity.com/paper/1904.02000/) (2019)
- [Cross-Target Stance Classification with Self-Attention Networks](https://mlsanity.com/paper/1805.06593/) (2018)
- [A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives](https://mlsanity.com/paper/2102.12982/) (2021)
- [Machine-Made Media: Monitoring the Mobilization of Machine-Generated Articles on Misinformation and Mainstream News Websites](https://mlsanity.com/paper/2305.09820/) (2023)
- [Adversarial Learning for Zero-Shot Stance Detection on Social Media](https://mlsanity.com/paper/2105.06603/) (2021)
- [A Review on Fact Extraction and Verification](https://mlsanity.com/paper/2010.03001/) (2020)
- [Contrastive Language Adaptation for Cross-Lingual Stance Detection](https://mlsanity.com/paper/1910.02076/) (2019)
- [Topic-Aware Evidence Reasoning and Stance-Aware Aggregation for Fact Verification](https://mlsanity.com/paper/2106.01191/) (2021)
- [Automatic Stance Detection Using End-to-End Memory Networks](https://mlsanity.com/paper/1804.07581/) (2018)

## Built on by (most cited first, 4 of 4)

- [CoSD: Collaborative Stance Detection with Contrastive Heterogeneous Topic Graph Learning](https://mlsanity.com/paper/2404.17609/) (2024)
- [Tracking the Takes and Trajectories of English-Language News Narratives across Trustworthy and Worrisome Websites](https://mlsanity.com/paper/2501.09102/) (2025)
- [Emotion-aware Dual Cross-Attentive Neural Network with Label Fusion for Stance Detection in Misinformative Social Media Content](https://mlsanity.com/paper/2505.23812/) (2025)
- [Zero-Shot Stance Detection in the Wild: Dynamic Target Generation and Multi-Target Adaptation](https://mlsanity.com/paper/2601.19802/) (2026)

## Abstract (arXiv)

Stance detection is important for understanding different attitudes and beliefs on the Internet. However, given that a passage's stance toward a given topic is often highly dependent on that topic, building a stance detection model that generalizes to unseen topics is difficult. In this work, we propose using contrastive learning as well as an unlabeled dataset of news articles that cover a variety of different topics to train topic-agnostic/TAG and topic-aware/TAW embeddings for use in downstream stance detection. Combining these embeddings in our full TATA model, we achieve state-of-the-art performance across several public stance detection datasets (0.771 F1-score on the Zero-shot VAST dataset). We release our code and data at https://github.com/hanshanley/tata.