mlsanityData to 7 Oct 2026

Applications and Techniques for Fast Machine Learning in Science

arXiv 2110.13041 ↗First posted 25 Oct 2021cs.LG cs.AR physics.data-ancited by 32

Not on the map. Why?

Abstract (arXiv)

In this community review report, we discuss applications and techniques for fast machine learning (ML) in science -- the concept of integrating power ML methods into the real-time experimental data processing loop to accelerate scientific discovery. The material for the report builds on two workshops held by the Fast ML for Science community and covers three main areas: applications for fast ML across a number of scientific domains; techniques for training and implementing performant and resource-efficient ML algorithms; and computing architectures, platforms, and technologies for deploying these algorithms. We also present overlapping challenges across the multiple scientific domains where common solutions can be found. This community report is intended to give plenty of examples and inspiration for scientific discovery through integrated and accelerated ML solutions. This is followed by a high-level overview and organization of technical advances, including an abundance of pointers to source material, which can enable these breakthroughs.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 171 of its 584 references that are in the map. First posted on arXiv 25 Oct 2021.

Builds on

171 of 584 references.

References · 171

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2016Densely Connected Convolutional Networkscited by 6,691
  8. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  9. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  10. 2017Squeeze-and-Excitation Networkscited by 4,763
  11. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  12. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
38 more
  1. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  2. 2019Searching for MobileNetV3cited by 1,984
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  4. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  5. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  7. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  8. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  9. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  10. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  11. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  12. 2019Relational Knowledge Distillationcited by 873
  13. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  14. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  15. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  16. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  17. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  18. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  19. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  20. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  21. 2020The Future of Digital Health with Federated Learningcited by 625
  22. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  23. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
  24. 2020What is the State of Neural Network Pruning?cited by 578
  25. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  26. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  27. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
  28. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 503
  29. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
  30. 2019The State of Sparsity in Deep Neural Networkscited by 471
  31. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 470
  32. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  33. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  34. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  35. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  36. 2018Shampoo: Preconditioned Stochastic Tensor Optimizationcited by 413
  37. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  38. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375

50 most cited of 171 All 171 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 9; 2023: 3; 2024: 7; 2025: 4; 2026: 7.

32 papers.

Sort:

  1. 2021Machine Learning in the Search for New Fundamental Physicscited by 50
  2. 2024HyperFast: Instant Classification for Tabular Datacited by 22
  3. 2022FastML Science Benchmarks: Accelerating Real-Time Scientific Edge Machine Learningcited by 12
  4. 2022Open-source FPGA-ML codesign for the MLPerf Tiny Benchmarkcited by 9
  5. 2024HGQ: High Granularity Quantization for Real-time Neural Networks on FPGAscited by 9
  6. 2025hls4ml: A Flexible, Open-Source Platform for Deep Learning Acceleration on Reconfigurable Hardwarecited by 8
  7. 2023Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamakcited by 7
  8. 2025Prior-Fitted Networks Scale to Larger Datasets When Treated as Weak Learnerscited by 7
  9. 2023Confident Feature Rankingcited by 6
  10. 2026Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategiescited by 6
  11. 2025Neural Architecture Codesign for Fast Physics Applicationscited by 5
  12. 2021Machine Learning in Nuclear Physicscited by 4
20 more
  1. 2022MLGWSC-1: The first Machine Learning Gravitational-Wave Search Mock Data Challengecited by 4
  2. 2022Do graph neural networks learn traditional jet substructure?cited by 4
  3. 2022Data Science and Machine Learning in Educationcited by 3
  4. 2022Innovations in trigger and data acquisition systems for next-generation physics facilitiescited by 2
  5. 2022Physics Community Needs, Tools, and Resources for Machine Learningcited by 2
  6. 2024Reliable edge machine learning hardware for scientific applicationscited by 2
  7. 2024CGRA4ML: A Hardware/Software Framework to Implement Neural Networks for Scientific Edge Computingcited by 2
  8. 2024Analysis of Hardware Synthesis Strategies for Machine Learning in Collider Trigger and Data Acquisitioncited by 2
  9. 2026Hardware-Aware Tensor Networks for Real-Time Quantum-Inspired Anomaly Detection at Particle Colliderscited by 2
  10. 2023End-to-end codesign of Hessian-aware quantized neural networks for FPGAs and ASICscited by 1
  11. 2024Embedded FPGA Developments in 130nm and 28nm CMOS for Machine Learning in Particle Detector Readoutcited by 1
  12. 2026A Lightweight Foundation Model for Collider Physics with Multi-Domain Adaptationcited by 1
  13. 2022Neural network accelerator for quantum controlcited by 0
  14. 2022Towards Robust Deep Learning using Entropic Lossescited by 0
  15. 2024Automated and Holistic Co-design of Neural Networks and ASICs for Enabling In-Pixel Intelligencecited by 0
  16. 2025Learning Before Filtering: Real-Time Hardware Learning at the Detector Levelcited by 0
  17. 2026Reconfigurable Computing Challenge: Real-Time Graph Neural Networks for Online Event Selection in Big Sciencecited by 0
  18. 2026Patch Hierarchical Attention Transformer for Efficient Particle Jet Taggingcited by 0
  19. 2026Toward Trustworthy Autonomous Science: A Two-Year Community Roadmapcited by 0
  20. 2026Real-Time Plasma State Prediction via FPGA-Accelerated Quantized Recurrent Probabilistic Neural Networkscited by 0