# Myths and Legends in High-Performance Computing

arXiv: https://arxiv.org/abs/2301.02432 · HTML: https://mlsanity.com/paper/2301.02432/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 73 references that are in the map.
- First posted on arXiv 6 Jan 2023.

## Builds on (most cited first, 20 of 20)

- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast](https://mlsanity.com/paper/2211.02556/) (2022)
- [Deep learning to represent sub-grid processes in climate models](https://mlsanity.com/paper/1806.04731/) (2018)
- [Data Movement Is All You Need: A Case Study on Optimizing Transformers](https://mlsanity.com/paper/2007.00072/) (2020)
- [AI Poincar'e: Machine Learning Conservation Laws from Trajectories](https://mlsanity.com/paper/2011.04698/) (2020)
- [Simulation Intelligence: Towards a New Generation of Scientific Methods](https://mlsanity.com/paper/2112.03235/) (2021)
- [Building high accuracy emulators for scientific simulations with deep neural architecture search](https://mlsanity.com/paper/2001.08055/) (2020)
- [Compressing multidimensional weather and climate data into neural networks](https://mlsanity.com/paper/2210.12538/) (2022)
- [Stateful Dataflow Multigraphs: A Data-Centric Model for Performance Portability on Heterogeneous Architectures](https://mlsanity.com/paper/1902.10345/) (2019)
- [Neural Networks are Decision Trees](https://mlsanity.com/paper/2210.05189/) (2022)
- [VerifyMed -- A blockchain platform for transparent trust in virtualized healthcare: Proof-of-concept](https://mlsanity.com/paper/2005.08804/) (2020)
- [Assessing requirements to scale to practical quantum advantage](https://mlsanity.com/paper/2211.07629/) (2022)
- [Disaggregated and optically interconnected memory: when will it be cost effective?](https://mlsanity.com/paper/1503.01416/) (2015)
- [Optically Connected Memory for Disaggregated Data Centers](https://mlsanity.com/paper/2008.10802/) (2020)
- [Matrix Engines for High Performance Computing:A Paragon of Performance or Grasping at Straws?](https://mlsanity.com/paper/2010.14373/) (2020)
- [Productive Performance Engineering for Weather and Climate Modeling with Python](https://mlsanity.com/paper/2205.04148/) (2022)
- [Noise in the Clouds: Influence of Network Performance Variability on Application Scalability](https://mlsanity.com/paper/2210.15315/) (2022)
- [More Bang for Your Buck: Improved use of GPU Nodes for GROMACS 2018](https://mlsanity.com/paper/1903.05918/) (2019)
- [Recovering single precision accuracy from Tensor Cores while surpassing the FP32 theoretical peak performance](https://mlsanity.com/paper/2203.03341/) (2022)

## Abstract (arXiv)

In this thought-provoking article, we discuss certain myths and legends that are folklore among members of the high-performance computing community. We gathered these myths from conversations at conferences and meetings, product advertisements, papers, and other communications such as tweets, blogs, and news articles within and beyond our community. We believe they represent the zeitgeist of the current era of massive change, driven by the end of many scaling laws such as Dennard scaling and Moore's law. While some laws end, new directions are emerging, such as algorithmic scaling or novel architecture research. Nevertheless, these myths are rarely based on scientific facts, but rather on some evidence or argumentation. In fact, we believe that this is the very reason for the existence of many myths and why they cannot be answered clearly. While it feels like there should be clear answers for each, some may remain endless philosophical debates, such as whether Beethoven was better than Mozart. We would like to see our collection of myths as a discussion of possible new directions for research and industry investment.