mlsanityData to 14 Oct 2026

Automated and Autonomous Experiment in Electron and Scanning Probe Microscopy

arXiv 2103.12165 ↗First posted 22 Mar 2021cs.LG cond-mat.mtrl-scicited by 1

Not on the map. Why?

Abstract (arXiv)

Machine learning and artificial intelligence (ML/AI) are rapidly becoming an indispensable part of physics research, with domain applications ranging from theory and materials prediction to high-throughput data analysis. In parallel, the recent successes in applying ML/AI methods for autonomous systems from robotics through self-driving cars to organic and inorganic synthesis are generating enthusiasm for the potential of these techniques to enable automated and autonomous experiment (AE) in imaging. Here, we aim to analyze the major pathways towards AE in imaging methods with sequential image formation mechanisms, focusing on scanning probe microscopy (SPM) and (scanning) transmission electron microscopy ((S)TEM). We argue that automated experiments should necessarily be discussed in a broader context of the general domain knowledge that both informs the experiment and is increased as the result of the experiment. As such, this analysis should explore the human and ML/AI roles prior to and during the experiment, and consider the latencies, biases, and knowledge priors of the decision-making process. Similarly, such discussion should include the limitations of the existing imaging systems, including intrinsic latencies, non-idealities and drifts comprising both correctable and stochastic components. We further pose that the role of the AE in microscopy is not the exclusion of human operators (as is the case for autonomous driving), but rather automation of routine operations such as microscope tuning, etc., prior to the experiment, and conversion of low latency decision making processes on the time scale spanning from image acquisition to human-level high-order experiment planning.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 166 references that are in the map. First posted on arXiv 22 Mar 2021.

Builds on

27 of 166 references.

References · 27

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  3. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  6. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  7. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  8. 2018Learning Dexterous In-Hand Manipulationcited by 687
  9. 2018Reward learning from human preferences and demonstrations in Ataricited by 316
  10. 2018Towards a Definition of Disentangled Representationscited by 296
  11. 2019Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networkscited by 200
  12. 2015Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)cited by 177
15 more
  1. 2013Gaussian Process Kernels for Pattern Discovery and Extrapolationcited by 132
  2. 2018Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applicationscited by 127
  3. 2019Deep neural network solution of the electronic Schr"odinger equationcited by 127
  4. 2018Bias and Generalization in Deep Generative Models: An Empirical Studycited by 83
  5. 2020Learning Invariances in Neural Networkscited by 52
  6. 2017Stein Variational Policy Gradientcited by 44
  7. 2020Machine learning for electronically excited states of moleculescited by 30
  8. 2019Explicitly disentangling image content from translation and rotation with spatial-VAEcited by 28
  9. 2015Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equationcited by 25
  10. 2014Hyperspectral and Multispectral Image Fusion based on a Sparse Representationcited by 23
  11. 2017Deep Rotation Equivariant Networkcited by 16
  12. 2013Bayesian Fusion of Multi-Band Imagescited by 13
  13. 2018A unified strategy for implementing curiosity and empowerment driven reinforcement learningcited by 9
  14. 2019Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernelscited by 6
  15. 2021Ensemble learning and iterative training (ELIT) machine learning: applications towards uncertainty quantification and automated experiment in atom-resolved microscopycited by 3

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Deep Reinforcement Learning for Data-Driven Adaptive Scanning in Ptychographycited by 3