Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Development
Not on the map. Why?
Abstract (arXiv)
Practitioners from diverse occupations and backgrounds are increasingly using machine learning (ML) methods. Nonetheless, studies on ML Practitioners typically draw populations from Big Tech and academia, as researchers have easier access to these communities. Through this selection bias, past research often excludes the broader, lesser-resourced ML community -- for example, practitioners working at startups, at non-tech companies, and in the public sector. These practitioners share many of the same ML development difficulties and ethical conundrums as their Big Tech counterparts; however, their experiences are subject to additional under-studied challenges stemming from deploying ML with limited resources, increased existential risk, and absent access to in-house research teams. We contribute a qualitative analysis of 17 interviews with stakeholders from organizations which are less represented in prior studies. We uncover a number of tensions which are introduced or exacerbated by these organizations' resource constraints -- tensions between privacy and ubiquity, resource management and performance optimization, and access and monopolization. Increased academic focus on these practitioners can facilitate a more holistic understanding of ML limitations, and so is useful for prescribing a research agenda to facilitate responsible ML development for all.
Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 19 of its 61 references that are in the map. First posted on arXiv 6 Oct 2021.
Builds on
19 of 61 references.
References · 19
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
- 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
- 2018Model Cards for Model Reportingcited by 1,080
- 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
- 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 496
- 2019Making AI Forget You: Data Deletion in Machine Learningcited by 465
- 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 384
- 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 344
- 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
- 2020Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertaintycited by 129
7 more
- 2018The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standardscited by 97
- 2019Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AIcited by 79
- 2016Decentralized Collaborative Learning of Personalized Models over Networkscited by 76
- 2020Human Factors in Model Interpretability: Industry Practices, Challenges, and Needscited by 74
- 2019Putting Fairness Principles into Practice: Challenges, Metrics, and Improvementscited by 70
- 2020How do Data Science Workers Collaborate? Roles, Workflows, and Toolscited by 64
- 2020Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Makingcited by 25
Built on by
Citing papers per year: 2021: 1; 2022: 2; 2023: 6; 2024: 9; 2025: 1; 2026: 2.
21 papers.
Sort:
- 2021Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Processcited by 56
- 2023Improving Human-AI Collaboration With Descriptions of AI Behaviorcited by 36
- 2023A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitionerscited by 22
- 2023Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiencescited by 21
- 2022Aspirations and Practice of Model Documentation: Moving the Needle with Nudging and Traceabilitycited by 14
- 2023A Systematic Literature Review of Human-Centered, Ethical, and Responsible AIcited by 6
- 2024A Large-Scale Study of Model Integration in ML-Enabled Software Systemscited by 6
- 2024From Silos to Systems: Process-Oriented Hazard Analysis for AI Systemscited by 6
- 2022Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Expertscited by 4
- 2025What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patternscited by 4
- 2024Data Ethics Emergency Drill: A Toolbox for Discussing Responsible AI for Industry Teamscited by 3
- 2024XAIport: A Service Framework for the Early Adoption of XAI in AI Model Developmentcited by 3
9 more
- 2024The Implications of Open Generative Models in Human-Centered Data Science Work: A Case Study with Fact-Checking Organizationscited by 3
- 2023Designing Data: Proactive Data Collection and Iteration for Machine Learningcited by 2
- 2023Machine Learning practices and infrastructurescited by 2
- 2024A Conceptual Framework for Ethical Evaluation of Machine Learning Systemscited by 2
- 2026Engaged AI Governance: Addressing the Last Mile Challenge Through Internal Expert Collaborationcited by 2
- 2024Navigating Fairness: Practitioners' Understanding, Challenges, and Strategies in AI/ML Developmentcited by 1
- 2024Why Algorithms Remain Unjust: Power Structures Surrounding Algorithmic Activitycited by 0
- 2024Using Quality Attribute Scenarios for ML Model Test Case Generationcited by 0
- 2026What We Know about Responsible AI Practices in Industry: A Half Decade of Empirical Researchcited by 0
- 2026What We Know about Responsible AI Practices in Industry: A Half Decade of Empirical Research11 Aug 2026
- 2026Engaged AI Governance: Addressing the Last Mile Challenge Through Internal Expert Collaboration23 Apr 2026
- 2025What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patterns22 Apr 2025
- 2024From Silos to Systems: Process-Oriented Hazard Analysis for AI Systems29 Oct 2024
- 2024A Large-Scale Study of Model Integration in ML-Enabled Software Systems12 Aug 2024
- 2024A Conceptual Framework for Ethical Evaluation of Machine Learning Systems5 Aug 2024
- 2024The Implications of Open Generative Models in Human-Centered Data Science Work: A Case Study with Fact-Checking Organizations4 Aug 2024
- 2024Using Quality Attribute Scenarios for ML Model Test Case Generation12 Jun 2024
- 2024Why Algorithms Remain Unjust: Power Structures Surrounding Algorithmic Activity28 May 2024
- 2024XAIport: A Service Framework for the Early Adoption of XAI in AI Model Development25 Mar 2024
- 2024Navigating Fairness: Practitioners' Understanding, Challenges, and Strategies in AI/ML Development21 Mar 2024
- 2024Data Ethics Emergency Drill: A Toolbox for Discussing Responsible AI for Industry Teams15 Mar 2024
9 more
- 2023Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences6 Oct 2023
- 2023Machine Learning practices and infrastructures13 Jul 2023
- 2023A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners31 Mar 2023
- 2023A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI10 Feb 2023
- 2023Designing Data: Proactive Data Collection and Iteration for Machine Learning24 Jan 2023
- 2023Improving Human-AI Collaboration With Descriptions of AI Behavior6 Jan 2023
- 2022Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts11 May 2022
- 2022Aspirations and Practice of Model Documentation: Moving the Needle with Nudging and Traceability13 Apr 2022
- 2021Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process19 Oct 2021
- 2021Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Processcited by 56
- 2023Improving Human-AI Collaboration With Descriptions of AI Behaviorcited by 36
- 2023A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitionerscited by 22
- 2023Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiencescited by 21
- 2022Aspirations and Practice of Model Documentation: Moving the Needle with Nudging and Traceabilitycited by 14
- 2023A Systematic Literature Review of Human-Centered, Ethical, and Responsible AIcited by 6
- 2024A Large-Scale Study of Model Integration in ML-Enabled Software Systemscited by 6
- 2024From Silos to Systems: Process-Oriented Hazard Analysis for AI Systemscited by 6
- 2022Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Expertscited by 4
- 2025What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patternscited by 4
- 2024Data Ethics Emergency Drill: A Toolbox for Discussing Responsible AI for Industry Teamscited by 3
- 2024XAIport: A Service Framework for the Early Adoption of XAI in AI Model Developmentcited by 3
12 most cited of 21 All 21 (JSON)
- 2026What We Know about Responsible AI Practices in Industry: A Half Decade of Empirical Research11 Aug 2026
- 2026Engaged AI Governance: Addressing the Last Mile Challenge Through Internal Expert Collaboration23 Apr 2026
- 2025What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patterns22 Apr 2025
- 2024From Silos to Systems: Process-Oriented Hazard Analysis for AI Systems29 Oct 2024
- 2024A Large-Scale Study of Model Integration in ML-Enabled Software Systems12 Aug 2024
- 2024A Conceptual Framework for Ethical Evaluation of Machine Learning Systems5 Aug 2024
- 2024The Implications of Open Generative Models in Human-Centered Data Science Work: A Case Study with Fact-Checking Organizations4 Aug 2024
- 2024Using Quality Attribute Scenarios for ML Model Test Case Generation12 Jun 2024
- 2024Why Algorithms Remain Unjust: Power Structures Surrounding Algorithmic Activity28 May 2024
- 2024XAIport: A Service Framework for the Early Adoption of XAI in AI Model Development25 Mar 2024
- 2024Navigating Fairness: Practitioners' Understanding, Challenges, and Strategies in AI/ML Development21 Mar 2024
- 2024Data Ethics Emergency Drill: A Toolbox for Discussing Responsible AI for Industry Teams15 Mar 2024
12 newest of 21 All 21 (JSON)