mlsanityData to 14 Oct 2026

Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems

arXiv 2104.13386 ↗First posted 27 Apr 2021cs.LG cond-mat.dis-nn cs.ETcited by 128

Not on the map. Why?

Abstract (arXiv)

Deep neural networks have become a pervasive tool in science and engineering. However, modern deep neural networks' growing energy requirements now increasingly limit their scaling and broader use. We propose a radical alternative for implementing deep neural network models: Physical Neural Networks. We introduce a hybrid physical-digital algorithm called Physics-Aware Training to efficiently train sequences of controllable physical systems to act as deep neural networks. This method automatically trains the functionality of any sequence of real physical systems, directly, using backpropagation, the same technique used for modern deep neural networks. To illustrate their generality, we demonstrate physical neural networks with three diverse physical systems-optical, mechanical, and electrical. Physical neural networks may facilitate unconventional machine learning hardware that is orders of magnitude faster and more energy efficient than conventional electronic processors.

Key facts. Cited by 128 papers in the map (data through 14 Oct 2026). Builds on 33 of its 107 references that are in the map. First posted on arXiv 27 Apr 2021.

Builds on

33 of 107 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  5. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  6. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  7. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  8. 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
  9. 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 259
  10. 2018All-Optical Machine Learning Using Diffractive Deep Neural Networkscited by 177
  11. 2020The Hardware Lotterycited by 160
  12. 2016A Differentiable Physics Engine for Deep Learning in Roboticscited by 108
21 more
  1. 2016Why does deep and cheap learning work so well?cited by 100
  2. 2019Theoretical Issues in Deep Networks: Approximation, Optimization and Generalizationcited by 72
  3. 2020Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Biascited by 64
  4. 2020Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architecturescited by 56
  5. 2018Training of photonic neural networks through in situ backpropagationcited by 45
  6. 2019Wave Physics as an Analog Recurrent Neural Networkcited by 43
  7. 2021A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrayscited by 39
  8. 2020Surrogate gradients for analog neuromorphic computingcited by 38
  9. 2020Building high accuracy emulators for scientific simulations with deep neural architecture searchcited by 36
  10. 2020EqSpike: Spike-driven Equilibrium Propagation for Neuromorphic Implementationscited by 31
  11. 2017Reinforcement Learning in a large scale photonic Recurrent Neural Networkcited by 30
  12. 2020Survey of Machine Learning Acceleratorscited by 29
  13. 2021Self-learning Machines based on Hamiltonian Echo Backpropagationcited by 28
  14. 2020Equilibrium Propagation with Continual Weight Updatescited by 26
  15. 2016Lets keep it simple, Using simple architectures to outperform deeper and more complex architecturescited by 25
  16. 2017Vowel recognition with four coupled spin-torque nano-oscillatorscited by 22
  17. 2019Summarizing CPU and GPU Design Trends with Product Datacited by 10
  18. 2014Trainable and Dynamic Computing: Error Backpropagation through Physical Mediacited by 9
  19. 2019Kernel computations from large-scale random features obtained by Optical Processing Unitscited by 6
  20. 2020Reality-assisted evolution of soft robots through large-scale physical experimentation: a reviewcited by 4
  21. 2020Physical deep learning based on optimal control of dynamical systemscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 18; 2023: 25; 2024: 24; 2025: 28; 2026: 31.

128 papers.

Sort:

By area:

  1. 2024Training of Physical Neural Networkscited by 49
  2. 2022Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networkscited by 44
  3. 2023Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based acceleratorscited by 43
  4. 2023Backpropagation-free Training of Deep Physical Neural Networkscited by 42
  5. 2022Learning on Arbitrary Graph Topologies via Predictive Codingcited by 33
  6. 2023Training an Ising Machine with Equilibrium Propagationcited by 29
  7. 2023Nonlinear Processing with Linear Opticscited by 20
  8. 2023Machine Learning Without a Processor: Emergent Learning in a Nonlinear Electronic Metamaterialcited by 20
  9. 2023Forward-Forward Training of an Optical Neural Networkcited by 19
  10. 2023Neural Oscillators are Universalcited by 16
  11. 2022Hybrid training of optical neural networkscited by 15
  12. 2022Agnostic Physics-Driven Deep Learningcited by 15
38 more
  1. 2023Thermodynamic AI and the fluctuation frontiercited by 15
  2. 2023Nonlinear optical encoding enabled by recurrent linear scatteringcited by 15
  3. 2021A compact butterfly-style silicon photonic-electronic neural chip for hardware-efficient deep learningcited by 13
  4. 2022Learning by non-interfering feedback chemical signaling in physical networkscited by 11
  5. 2022Frequency propagation: Multi-mechanism learning in nonlinear physical networkscited by 11
  6. 2024Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plantscited by 11
  7. 2022Polarization Multiplexed Diffractive Computing: All-Optical Implementation of a Group of Linear Transformations Through a Polarization-Encoded Diffractive Networkcited by 10
  8. 2022Dual adaptive training of photonic neural networkscited by 9
  9. 2024Arbitrary control over multimode wave propagation for machine learningcited by 9
  10. 2024Quantum Equilibrium Propagation for efficient training of quantum systems based on Onsager reciprocitycited by 9
  11. 2023Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagationcited by 8
  12. 2023Quantum-limited stochastic optical neural networks operating at a few quanta per activationcited by 8
  13. 2024Optical Generative Modelscited by 8
  14. 2023Training neural networks with end-to-end optical backpropagationcited by 7
  15. 2024A Pontryagin Perspective on Reinforcement Learningcited by 7
  16. 2024Towards Exact Gradient-based Training on Analog In-memory Computingcited by 7
  17. 2025Towards the Training of Deeper Predictive Coding Neural Networkscited by 7
  18. 2023A universal approximation theorem for nonlinear resistive networkscited by 6
  19. 2024A Review of Neuroscience-Inspired Machine Learningcited by 6
  20. 2022Data Models for Dataset Drift Controls in Machine Learning With Optical Imagescited by 5
  21. 2024Noise-Aware Training of Neuromorphic Dynamic Device Networkscited by 5
  22. 2024Training all-mechanical neural networks for task learning through in situ backpropagationcited by 5
  23. 2025Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functionscited by 5
  24. 2022Noise mitigation strategies in physical feedforward neural networkscited by 4
  25. 2023Learning Reservoir Dynamics with Temporal Self-Modulationcited by 4
  26. 2023Integrated multi-operand optical neurons for scalable and hardware-efficient deep learningcited by 4
  27. 2024Optical Diffusion Models for Image Generationcited by 4
  28. 2022Data-driven Modeling of Mach-Zehnder Interferometer-based Optical Matrix Multiplierscited by 3
  29. 2022Privacy-Enhancing Optical Embeddings for Lensless Classificationcited by 3
  30. 2023Measurement-driven neural-network training for integrated magnetic tunnel junction arrayscited by 3
  31. 2024Asymmetrical estimator for training encapsulated deep photonic neural networkscited by 3
  32. 2024Streamlined optical training of large-scale modern deep learning architectures with direct feedback alignmentcited by 3
  33. 2025Fisher information flow in artificial neural networkscited by 3
  34. 2025Bayesian Physics Informed Neural Networks for Reliable Transformer Prognosticscited by 3
  35. 2025Near-Equilibrium Propagation training in nonlinear wave systemscited by 3
  36. 2025Uncertainty Reasoning with Photonic Bayesian Machinescited by 3
  37. 2022Bias-Scalable Near-Memory CMOS Analog Processor for Machine Learningcited by 2
  38. 2022RF-Photonic Deep Learning Processor with Shannon-Limited Data Movementcited by 2

50 most cited of 128 All 128 (JSON)