Attention to Non-Adopters
Not on the map. Why?
Abstract (arXiv)
Although language model-based chat systems are increasingly used in daily life, most Americans remain non-adopters of chat-based LLMs -- as of June 2025, 66% had never used ChatGPT. At the same time, LLM development and evaluation rely mainly on data from adopters (e.g., logs, preference data), focusing on the needs and tasks for a limited demographic group of adopters in terms of geographic location, education, and gender. In this position paper, we argue that incorporating non-adopter perspectives is essential for developing broadly useful and capable LLMs. We contend that relying on methods that focus primarily on adopters will risk missing a range of tasks and needs prioritized by non-adopters, entrenching inequalities in who benefits from LLMs, and creating oversights in model development and evaluation. To illustrate this claim, we conduct case studies with non-adopters and show: how non-adopter needs diverge from those of current users, how non-adopter needs point us towards novel reasoning tasks, and how to systematically integrate non-adopter needs via human-centered methods.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 53 of its 154 references that are in the map. First posted on arXiv 10 Oct 2025.
Builds on
53 of 154 references.
References · 53
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
- 2022Competition-Level Code Generation with AlphaCodecited by 1,427
- 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
- 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
- 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
- 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
38 more
- 2023Whose Opinions Do Language Models Reflect?cited by 759
- 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
- 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
- 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
- 2022CodeT: Code Generation with Generated Testscited by 369
- 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agentscited by 236
- 2020IntelliCode Compose: Code Generation Using Transformercited by 225
- 2016Demographic Dialectal Variation in Social Media: A Case Study of African-American Englishcited by 194
- 2021The Values Encoded in Machine Learning Researchcited by 172
- 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
- 2022Power to the People? Opportunities and Challenges for Participatory AIcited by 137
- 2017DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applicationscited by 110
- 2020Participation is not a Design Fix for Machine Learningcited by 110
- 2023A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writingcited by 98
- 2024MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualizationcited by 98
- 2023Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Codingcited by 82
- 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUIcited by 81
- 2017Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American Englishcited by 69
- 2024Clio: Privacy-Preserving Insights into Real-World AI Usecited by 66
- 2023When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Datasetcited by 51
- 2021Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviorscited by 47
- 2024ExeGPT: Constraint-Aware Resource Scheduling for LLM Inferencecited by 43
- 2024The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labelscited by 37
- 2024Participation in the age of foundation modelscited by 32
- 2023Probing the Creativity of Large Language Models: Can models produce divergent semantic association?cited by 31
- 2022Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practicecited by 30
- 2022Whose AI Dream? In search of the aspiration in data annotationcited by 24
- 2025User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signalcited by 19
- 2023Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experiencecited by 18
- 2019Deep Conversational Recommender in Travelcited by 16
- 2021SHARE: a System for Hierarchical Assistive Recipe Editingcited by 15
- 2024Counterspeakers' Perspectives: Unveiling Barriers and AI Needs in the Fight against Online Hatecited by 14
- 2023Algorithmic Pluralism: A Structural Approach To Equal Opportunitycited by 13
- 2024Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefitscited by 9
- 2024Can GPT Improve the State of Prior Authorization via Guideline Based Automated Question Answering?cited by 7
- 2024From Fitting Participation to Forging Relationships: The Art of Participatory MLcited by 7
- 2024Catalyzing Equity in STEM Teams: Harnessing Generative AI for Inclusion and Diversitycited by 6
- 2024Policy Prototyping for LLMs: Pluralistic Alignment via Interactive and Collaborative Policymakingcited by 6
50 most cited of 53 All 53 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.
- 2026Real-World AI Evaluation: How FRAME Generates Systematic Evidence to Resolve the Decision-Maker's Dilemma28 Feb 2026
- 2026CIRCLE: A Framework for Evaluating AI from a Real-World Lens27 Feb 2026
- 2025Generative AI Practices, Literacy, and Divides: An Empirical Analysis in the Italian Context3 Dec 2025