# Bridging Data Center AI Systems with Edge Computing for Actionable Information Retrieval

arXiv: https://arxiv.org/abs/2105.13967 · HTML: https://mlsanity.com/paper/2105.13967/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 46 references that are in the map.
- First posted on arXiv 28 May 2021.

## Builds on (most cited first, 15 of 15)

- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Horovod: fast and easy distributed deep learning in TensorFlow](https://mlsanity.com/paper/1802.05799/) (2018)
- [Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads](https://mlsanity.com/paper/2105.11118/) (2021)
- [funcX: A Federated Function Serving Fabric for Science](https://mlsanity.com/paper/2005.04215/) (2020)
- [TomoGAN: Low-Dose Synchrotron X-Ray Tomography with Generative Adversarial Networks](https://mlsanity.com/paper/1902.07582/) (2019)
- [BraggNN: Fast X-ray Bragg Peak Analysis Using Deep Learning](https://mlsanity.com/paper/2008.08198/) (2020)
- [Fast Stencil-Code Computation on a Wafer-Scale Processor](https://mlsanity.com/paper/2010.03660/) (2020)
- [Joint ptycho-tomography with deep generative priors](https://mlsanity.com/paper/2009.09498/) (2020)
- [Machine learning applied to single-shot x-ray diagnostics in an XFEL](https://mlsanity.com/paper/1610.03378/) (2016)
- [Deep Learning Accelerated Light Source Experiments](https://mlsanity.com/paper/1910.04081/) (2019)
- [Scientific Image Restoration Anywhere](https://mlsanity.com/paper/1911.05878/) (2019)
- [DeepFreak: Learning Crystallography Diffraction Patterns with Automated Machine Learning](https://mlsanity.com/paper/1904.11834/) (2019)
- [Deep Learning-based Low-dose Tomography Reconstruction with Hybrid-dose Measurements](https://mlsanity.com/paper/2009.13589/) (2020)
- [Design and Evaluation of a Simple Data Interface for Efficient Data Transfer Across Diverse Storage](https://mlsanity.com/paper/2009.03190/) (2020)
- [Toward Real-time Analysis of Experimental Science Workloads on Geographically Distributed Supercomputers](https://mlsanity.com/paper/2105.06571/) (2021)

## Built on by (most cited first, 7 of 7)

- [SMLT: A Serverless Framework for Scalable and Adaptive Machine Learning Design and Training](https://mlsanity.com/paper/2205.01853/) (2022)
- [Globus Automation Services: Research process automation across the space-time continuum](https://mlsanity.com/paper/2208.09513/) (2022)
- [fairDMS: Rapid Model Training by Data and Model Reuse](https://mlsanity.com/paper/2204.09805/) (2022)
- [Masked Sinogram Model with Transformer for ill-Posed Computed Tomography Reconstruction: a Preliminary Study](https://mlsanity.com/paper/2209.01356/) (2022)
- [Ultrafast Focus Detection for Automated Microscopy](https://mlsanity.com/paper/2108.12050/) (2021)
- [Linking the Dynamic PicoProbe Analytical Electron-Optical Beam Line / Microscope to Supercomputers](https://mlsanity.com/paper/2308.13701/) (2023)
- [Neural Architecture Codesign for Fast Bragg Peak Analysis](https://mlsanity.com/paper/2312.05978/) (2023)

## Abstract (arXiv)

Extremely high data rates at modern synchrotron and X-ray free-electron laser light source beamlines motivate the use of machine learning methods for data reduction, feature detection, and other purposes. Regardless of the application, the basic concept is the same: data collected in early stages of an experiment, data from past similar experiments, and/or data simulated for the upcoming experiment are used to train machine learning models that, in effect, learn specific characteristics of those data; these models are then used to process subsequent data more efficiently than would general-purpose models that lack knowledge of the specific dataset or data class. Thus, a key challenge is to be able to train models with sufficient rapidity that they can be deployed and used within useful timescales. We describe here how specialized data center AI (DCAI) systems can be used for this purpose through a geographically distributed workflow. Experiments show that although there are data movement cost and service overhead to use remote DCAI systems for DNN training, the turnaround time is still less than 1/30 of using a locally deploy-able GPU.