# The Implications of Open Generative Models in Human-Centered Data Science Work: A Case Study with Fact-Checking Organizations

arXiv: https://arxiv.org/abs/2408.01962 · HTML: https://mlsanity.com/paper/2408.01962/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 60 references that are in the map.
- First posted on arXiv 4 Aug 2024.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [The Values Encoded in Machine Learning Research](https://mlsanity.com/paper/2106.15590/) (2021)
- [Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI](https://mlsanity.com/paper/1909.02309/) (2019)
- [How do Data Science Workers Collaborate? Roles, Workflows, and Tools](https://mlsanity.com/paper/2001.06684/) (2020)
- [The Gradient of Generative AI Release: Methods and Considerations](https://mlsanity.com/paper/2302.04844/) (2023)
- [Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Development](https://mlsanity.com/paper/2110.02932/) (2021)
- [Automated Claim Matching with Large Language Models: Empowering Fact-Checkers in the Fight Against Misinformation](https://mlsanity.com/paper/2310.09223/) (2023)
- [Modeling Legal Reasoning: LM Annotation at the Edge of Human Agreement](https://mlsanity.com/paper/2310.18440/) (2023)
- [Human and technological infrastructures of fact-checking](https://mlsanity.com/paper/2205.10894/) (2022)
- [The Impact and Opportunities of Generative AI in Fact-Checking](https://mlsanity.com/paper/2405.15985/) (2024)

## Built on by (most cited first, 3 of 3)

- [Beyond Release: Access Considerations for Generative AI Systems](https://mlsanity.com/paper/2502.16701/) (2025)
- [Fragments to Facts: Partial-Information Fragment Inference from LLMs](https://mlsanity.com/paper/2505.13819/) (2025)
- [Ctrl-F-Resist. Practices, Challenges, and Technical Needs of Civil Society Organizations Monitoring the Far-Right Online](https://mlsanity.com/paper/2609.00808/) (2026)

## Abstract (arXiv)

Calls to use open generative language models in academic research have highlighted the need for reproducibility and transparency in scientific research. However, the impact of generative AI extends well beyond academia, as corporations and public interest organizations have begun integrating these models into their data science pipelines. We expand this lens to include the impact of open models on organizations, focusing specifically on fact-checking organizations, which use AI to observe and analyze large volumes of circulating misinformation, yet must also ensure the reproducibility and impartiality of their work. We wanted to understand where fact-checking organizations use open models in their data science pipelines; what motivates their use of open models or proprietary models; and how their use of open or proprietary models can inform research on the societal impact of generative AI. To answer these questions, we conducted an interview study with N=24 professionals at 20 fact-checking organizations on six continents. Based on these interviews, we offer a five-component conceptual model of where fact-checking organizations employ generative AI to support or automate parts of their data science pipeline, including Data Ingestion, Data Analysis, Data Retrieval, Data Delivery, and Data Sharing. We then provide taxonomies of fact-checking organizations' motivations for using open models and the limitations that prevent them for further adopting open models, finding that they prefer open models for Organizational Autonomy, Data Privacy and Ownership, Application Specificity, and Capability Transparency. However, they nonetheless use proprietary models due to perceived advantages in Performance, Usability, and Safety, as well as Opportunity Costs related to participation in emerging generative AI ecosystems. Our work provides novel perspective on open models in data-driven organizations.