# The Grand Illusion: The Myth of Software Portability and Implications for ML Progress

arXiv: https://arxiv.org/abs/2309.07181 · HTML: https://mlsanity.com/paper/2309.07181/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 17 of 17)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [MLPerf Inference Benchmark](https://mlsanity.com/paper/1911.02549/) (2019)
- [MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems](https://mlsanity.com/paper/1512.01274/) (2015)
- [The Hardware Lottery](https://mlsanity.com/paper/2009.06489/) (2020)
- [Pathways: Asynchronous Distributed Dataflow for ML](https://mlsanity.com/paper/2203.12533/) (2022)
- [The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research](https://mlsanity.com/paper/2010.15581/) (2020)
- [Theano: A Python framework for fast computation of mathematical expressions](https://mlsanity.com/paper/1605.02688/) (2016)
- [Chainer: A Deep Learning Framework for Accelerating the Research Cycle](https://mlsanity.com/paper/1908.00213/) (2019)
- [Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training](https://mlsanity.com/paper/2006.03318/) (2020)
- [TSSB-3M: Mining single statement bugs at massive scale](https://mlsanity.com/paper/2201.12046/) (2022)
- [A narrowing of AI research?](https://mlsanity.com/paper/2009.10385/) (2020)

## Built on by (most cited first, 3 of 3)

- [On the Limitations of Compute Thresholds as a Governance Strategy](https://mlsanity.com/paper/2407.05694/) (2024)
- [On The Fairness Impacts of Hardware Selection in Machine Learning](https://mlsanity.com/paper/2312.03886/) (2023)
- [Deep Learning 2.0: Artificial Neurons That Matter -- Reject Correlation, Embrace Orthogonality](https://mlsanity.com/paper/2411.08085/) (2024)

## Abstract (arXiv)

Pushing the boundaries of machine learning often requires exploring different hardware and software combinations. However, the freedom to experiment across different tooling stacks can be at odds with the drive for efficiency, which has produced increasingly specialized AI hardware and incentivized consolidation around a narrow set of ML frameworks. Exploratory research can be restricted if software and hardware are co-evolving, making it even harder to stray away from mainstream ideas that work well with popular tooling stacks. While this friction increasingly impacts the rate of innovation in machine learning, to our knowledge the lack of portability in tooling has not been quantified. In this work, we ask: How portable are popular ML software frameworks? We conduct a large-scale study of the portability of mainstream ML frameworks across different hardware types. Our findings paint an uncomfortable picture -- frameworks can lose more than 40% of their key functions when ported to other hardware. Worse, even when functions are portable, the slowdown in their performance can be extreme and render performance untenable. Collectively, our results reveal how costly straying from a narrow set of hardware-software combinations can be - and suggest that specialization of hardware impedes innovation in machine learning research.