# Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Development

arXiv: https://arxiv.org/abs/2110.02932 · HTML: https://mlsanity.com/paper/2110.02932/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 61 references that are in the map.
- First posted on arXiv 6 Oct 2021.

## Builds on (most cited first, 19 of 19)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty](https://mlsanity.com/paper/2011.07586/) (2020)
- [The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards](https://mlsanity.com/paper/1805.03677/) (2018)
- [Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI](https://mlsanity.com/paper/1909.02309/) (2019)
- [Decentralized Collaborative Learning of Personalized Models over Networks](https://mlsanity.com/paper/1610.05202/) (2016)
- [Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs](https://mlsanity.com/paper/2004.11440/) (2020)
- [Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements](https://mlsanity.com/paper/1901.04562/) (2019)
- [How do Data Science Workers Collaborate? Roles, Workflows, and Tools](https://mlsanity.com/paper/2001.06684/) (2020)
- [Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making](https://mlsanity.com/paper/2005.10960/) (2020)

## Built on by (most cited first, 21 of 21)

- [Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process](https://mlsanity.com/paper/2110.10234/) (2021)
- [Improving Human-AI Collaboration With Descriptions of AI Behavior](https://mlsanity.com/paper/2301.06937/) (2023)
- [A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners](https://mlsanity.com/paper/2304.00078/) (2023)
- [Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences](https://mlsanity.com/paper/2310.04621/) (2023)
- [Aspirations and Practice of Model Documentation: Moving the Needle with Nudging and Traceability](https://mlsanity.com/paper/2204.06425/) (2022)
- [A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI](https://mlsanity.com/paper/2302.05284/) (2023)
- [A Large-Scale Study of Model Integration in ML-Enabled Software Systems](https://mlsanity.com/paper/2408.06226/) (2024)
- [From Silos to Systems: Process-Oriented Hazard Analysis for AI Systems](https://mlsanity.com/paper/2410.22526/) (2024)
- [Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts](https://mlsanity.com/paper/2205.05749/) (2022)
- [What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patterns](https://mlsanity.com/paper/2504.15815/) (2025)
- [Data Ethics Emergency Drill: A Toolbox for Discussing Responsible AI for Industry Teams](https://mlsanity.com/paper/2403.10438/) (2024)
- [XAIport: A Service Framework for the Early Adoption of XAI in AI Model Development](https://mlsanity.com/paper/2403.16858/) (2024)
- [The Implications of Open Generative Models in Human-Centered Data Science Work: A Case Study with Fact-Checking Organizations](https://mlsanity.com/paper/2408.01962/) (2024)
- [Designing Data: Proactive Data Collection and Iteration for Machine Learning](https://mlsanity.com/paper/2301.10319/) (2023)
- [Machine Learning practices and infrastructures](https://mlsanity.com/paper/2307.06518/) (2023)
- [A Conceptual Framework for Ethical Evaluation of Machine Learning Systems](https://mlsanity.com/paper/2408.10239/) (2024)
- [Engaged AI Governance: Addressing the Last Mile Challenge Through Internal Expert Collaboration](https://mlsanity.com/paper/2604.21554/) (2026)
- [Navigating Fairness: Practitioners' Understanding, Challenges, and Strategies in AI/ML Development](https://mlsanity.com/paper/2403.15481/) (2024)
- [Why Algorithms Remain Unjust: Power Structures Surrounding Algorithmic Activity](https://mlsanity.com/paper/2405.18461/) (2024)
- [Using Quality Attribute Scenarios for ML Model Test Case Generation](https://mlsanity.com/paper/2406.08575/) (2024)
- [What We Know about Responsible AI Practices in Industry: A Half Decade of Empirical Research](https://mlsanity.com/paper/2608.10431/) (2026)

## Abstract (arXiv)

Practitioners from diverse occupations and backgrounds are increasingly using machine learning (ML) methods. Nonetheless, studies on ML Practitioners typically draw populations from Big Tech and academia, as researchers have easier access to these communities. Through this selection bias, past research often excludes the broader, lesser-resourced ML community -- for example, practitioners working at startups, at non-tech companies, and in the public sector. These practitioners share many of the same ML development difficulties and ethical conundrums as their Big Tech counterparts; however, their experiences are subject to additional under-studied challenges stemming from deploying ML with limited resources, increased existential risk, and absent access to in-house research teams. We contribute a qualitative analysis of 17 interviews with stakeholders from organizations which are less represented in prior studies. We uncover a number of tensions which are introduced or exacerbated by these organizations' resource constraints -- tensions between privacy and ubiquity, resource management and performance optimization, and access and monopolization. Increased academic focus on these practitioners can facilitate a more holistic understanding of ML limitations, and so is useful for prescribing a research agenda to facilitate responsible ML development for all.