# Applications and Techniques for Fast Machine Learning in Science

arXiv: https://arxiv.org/abs/2110.13041 · HTML: https://mlsanity.com/paper/2110.13041/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 171 of its 584 references that are in the map.
- First posted on arXiv 25 Oct 2021.

## Builds on (most cited first, 30 of 171)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [SNIP: Single-shot Network Pruning based on Connection Sensitivity](https://mlsanity.com/paper/1810.02340/) (2018)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)

## Built on by (most cited first, 30 of 32)

- [Machine Learning in the Search for New Fundamental Physics](https://mlsanity.com/paper/2112.03769/) (2021)
- [HyperFast: Instant Classification for Tabular Data](https://mlsanity.com/paper/2402.14335/) (2024)
- [FastML Science Benchmarks: Accelerating Real-Time Scientific Edge Machine Learning](https://mlsanity.com/paper/2207.07958/) (2022)
- [Open-source FPGA-ML codesign for the MLPerf Tiny Benchmark](https://mlsanity.com/paper/2206.11791/) (2022)
- [HGQ: High Granularity Quantization for Real-time Neural Networks on FPGAs](https://mlsanity.com/paper/2405.00645/) (2024)
- [hls4ml: A Flexible, Open-Source Platform for Deep Learning Acceleration on Reconfigurable Hardware](https://mlsanity.com/paper/2512.01463/) (2025)
- [Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategies](https://mlsanity.com/paper/2603.16952/) (2026)
- [Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamak](https://mlsanity.com/paper/2312.00128/) (2023)
- [Prior-Fitted Networks Scale to Larger Datasets When Treated as Weak Learners](https://mlsanity.com/paper/2503.01256/) (2025)
- [Confident Feature Ranking](https://mlsanity.com/paper/2307.15361/) (2023)
- [Neural Architecture Codesign for Fast Physics Applications](https://mlsanity.com/paper/2501.05515/) (2025)
- [Machine Learning in Nuclear Physics](https://mlsanity.com/paper/2112.02309/) (2021)
- [MLGWSC-1: The first Machine Learning Gravitational-Wave Search Mock Data Challenge](https://mlsanity.com/paper/2209.11146/) (2022)
- [Do graph neural networks learn traditional jet substructure?](https://mlsanity.com/paper/2211.09912/) (2022)
- [Data Science and Machine Learning in Education](https://mlsanity.com/paper/2207.09060/) (2022)
- [Innovations in trigger and data acquisition systems for next-generation physics facilities](https://mlsanity.com/paper/2203.07620/) (2022)
- [Physics Community Needs, Tools, and Resources for Machine Learning](https://mlsanity.com/paper/2203.16255/) (2022)
- [End-to-end codesign of Hessian-aware quantized neural networks for FPGAs and ASICs](https://mlsanity.com/paper/2304.06745/) (2023)
- [Reliable edge machine learning hardware for scientific applications](https://mlsanity.com/paper/2406.19522/) (2024)
- [CGRA4ML: A Hardware/Software Framework to Implement Neural Networks for Scientific Edge Computing](https://mlsanity.com/paper/2408.15561/) (2024)
- [Analysis of Hardware Synthesis Strategies for Machine Learning in Collider Trigger and Data Acquisition](https://mlsanity.com/paper/2411.11678/) (2024)
- [Hardware-Aware Tensor Networks for Real-Time Quantum-Inspired Anomaly Detection at Particle Colliders](https://mlsanity.com/paper/2603.26604/) (2026)
- [Embedded FPGA Developments in 130nm and 28nm CMOS for Machine Learning in Particle Detector Readout](https://mlsanity.com/paper/2404.17701/) (2024)
- [A Lightweight Foundation Model for Collider Physics with Multi-Domain Adaptation](https://mlsanity.com/paper/2607.27501/) (2026)
- [Neural network accelerator for quantum control](https://mlsanity.com/paper/2208.02645/) (2022)
- [Towards Robust Deep Learning using Entropic Losses](https://mlsanity.com/paper/2208.03566/) (2022)
- [Automated and Holistic Co-design of Neural Networks and ASICs for Enabling In-Pixel Intelligence](https://mlsanity.com/paper/2407.14560/) (2024)
- [Learning Before Filtering: Real-Time Hardware Learning at the Detector Level](https://mlsanity.com/paper/2506.11981/) (2025)
- [Reconfigurable Computing Challenge: Real-Time Graph Neural Networks for Online Event Selection in Big Science](https://mlsanity.com/paper/2605.10612/) (2026)
- [Patch Hierarchical Attention Transformer for Efficient Particle Jet Tagging](https://mlsanity.com/paper/2605.21789/) (2026)

## Abstract (arXiv)

In this community review report, we discuss applications and techniques for fast machine learning (ML) in science -- the concept of integrating power ML methods into the real-time experimental data processing loop to accelerate scientific discovery. The material for the report builds on two workshops held by the Fast ML for Science community and covers three main areas: applications for fast ML across a number of scientific domains; techniques for training and implementing performant and resource-efficient ML algorithms; and computing architectures, platforms, and technologies for deploying these algorithms. We also present overlapping challenges across the multiple scientific domains where common solutions can be found. This community report is intended to give plenty of examples and inspiration for scientific discovery through integrated and accelerated ML solutions. This is followed by a high-level overview and organization of technical advances, including an abundance of pointers to source material, which can enable these breakthroughs.