# Circuit design in biology and machine learning. I. Random networks and dimensional reduction

arXiv: https://arxiv.org/abs/2408.09604 · HTML: https://mlsanity.com/paper/2408.09604/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 99 references that are in the map.
- First posted on arXiv 18 Aug 2024.

## Builds on (most cited first, 7 of 7)

- [Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training](https://mlsanity.com/paper/2305.14342/) (2023)
- [Next Generation Reservoir Computing](https://mlsanity.com/paper/2106.07688/) (2021)
- [A survey of dimensionality reduction techniques](https://mlsanity.com/paper/1403.2877/) (2014)
- [The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold](https://mlsanity.com/paper/2305.01604/) (2023)
- [ReservoirComputing.jl: An Efficient and Modular Library for Reservoir Computing Models](https://mlsanity.com/paper/2204.05117/) (2022)
- [DNA Reservoir Computing: A Novel Molecular Computing Approach](https://mlsanity.com/paper/1306.5998/) (2013)
- [Hybridizing Traditional and Next-Generation Reservoir Computing to Accurately and Efficiently Forecast Dynamical Systems](https://mlsanity.com/paper/2403.18953/) (2024)

## Built on by (most cited first, 1 of 1)

- [Circuit design in biology and machine learning. II. Anomaly detection](https://mlsanity.com/paper/2411.15647/) (2024)

## Abstract (arXiv)

A biological circuit is a neural or biochemical cascade, taking inputs and producing outputs. How have biological circuits learned to solve environmental challenges over the history of life? The answer certainly follows Dobzhansky's famous quote that ``nothing in biology makes sense except in the light of evolution.'' But that quote leaves out the mechanistic basis by which natural selection's trial-and-error learning happens, which is exactly what we have to understand. How does the learning process that designs biological circuits actually work? How much insight can we gain about the form and function of biological circuits by studying the processes that have made those circuits? Because life's circuits must often solve the same problems as those faced by machine learning, such as environmental tracking, homeostatic control, dimensional reduction, or classification, we can begin by considering how machine learning designs computational circuits to solve problems. We can then ask: How much insight do those computational circuits provide about the design of biological circuits? How much does biology differ from computers in the particular circuit designs that it uses to solve problems? This article steps through two classic machine learning models to set the foundation for analyzing broad questions about the design of biological circuits. One insight is the surprising power of randomly connected networks. Another is the central role of internal models of the environment embedded within biological circuits, illustrated by a model of dimensional reduction and trend prediction. Overall, many challenges in biology have machine learning analogs, suggesting hypotheses about how biology's circuits are designed.