# A Longitudinal Assessment of the Persistence of Twitter Datasets

arXiv: https://arxiv.org/abs/1709.09186 · HTML: https://mlsanity.com/paper/1709.09186/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2017.

## Built on by (most cited first, 11 of 11)

- [Cross-Domain Label-Adaptive Stance Detection](https://mlsanity.com/paper/2104.07467/) (2021)
- [Yes-Yes-Yes: Proactive Data Collection for ACL Rolling Review and Beyond](https://mlsanity.com/paper/2201.11443/) (2022)
- [CrowdChecked: Detecting Previously Fact-Checked Claims in Social Media](https://mlsanity.com/paper/2210.04447/) (2022)
- [Capturing Stance Dynamics in Social Media: Open Challenges and Research Directions](https://mlsanity.com/paper/2109.00475/) (2021)
- [Cross-Target Stance Detection: A Survey of Techniques, Datasets, and Challenges](https://mlsanity.com/paper/2409.13594/) (2024)
- [Decay No More: A Persistent Twitter Dataset for Learning Social Meaning](https://mlsanity.com/paper/2204.04611/) (2022)
- [The emojification of sentiment on social media: Collection and analysis of a longitudinal Twitter sentiment dataset](https://mlsanity.com/paper/2108.13898/) (2021)
- [SocialPET: Socially Informed Pattern Exploiting Training for Few-Shot Stance Detection in Social Media](https://mlsanity.com/paper/2403.05216/) (2024)
- [Zero-shot Cross-lingual Stance Detection via Adversarial Language Adaptation](https://mlsanity.com/paper/2404.14339/) (2024)
- [Enhancing Data Quality through Simple De-duplication: Navigating Responsible Computational Social Science Research](https://mlsanity.com/paper/2410.03545/) (2024)
- [Bots Don't Sit Still: A Longitudinal Study of Bot Behaviour Change, Temporal Drift, and Feature-Structure Evolution](https://mlsanity.com/paper/2512.17067/) (2025)

## Abstract (arXiv)

With social media datasets being increasingly shared by researchers, it also presents the caveat that those datasets are not always completely replicable. Having to adhere to requirements of platforms like Twitter, researchers cannot release the raw data and instead have to release a list of unique identifiers, which others can then use to recollect the data from the platform themselves. This leads to the problem that subsets of the data may no longer be available, as content can be deleted or user accounts deactivated. To quantify the impact of content deletion in the replicability of datasets in a long term, we perform a longitudinal analysis of the persistence of 30 Twitter datasets, which include over 147 million tweets. Having the original datasets collected between 2012 and 2016, and recollecting them later by using the tweet IDs, we look at four different factors that quantify the extent to which recollected datasets resemble original ones: completeness, representativity, similarity and changingness. Even though the ratio of available tweets keeps decreasing as the dataset gets older, we find that the textual content of the recollected subset is still largely representative of the whole dataset that was originally collected. The representativity of the metadata, however, keeps decreasing over time, both because the dataset shrinks and because certain metadata, such as the users' number of followers, keeps changing. Our study has important implications for researchers sharing and using publicly shared Twitter datasets in their research.