# Massively Parallel Reinforcement Learning with a Chaotic Reconfigurable Clockless Chip

arXiv: https://arxiv.org/abs/2609.36347 · HTML: https://mlsanity.com/paper/2609.36347/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Spiking and neuromorphic RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 7 of its 50 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Placed in

- Deep reinforcement learning methods / [Spiking and neuromorphic RL](https://mlsanity.com/deeprl/spiking-and-neuromorphic-rl/)

## Builds on (most cited first, 7 of 7)

- [Ultrafast photonic reinforcement learning based on laser chaos](https://mlsanity.com/paper/1704.04379/) (2017)
- [Efficient Decision-Making by Volume-Conserving Physical Object](https://mlsanity.com/paper/1412.6141/) (2014)
- [Scalable photonic reinforcement learning by time-division multiplexing of laser chaos](https://mlsanity.com/paper/1803.09425/) (2018)
- [Controlling chaotic itinerancy in laser dynamics for reinforcement learning](https://mlsanity.com/paper/2205.05987/) (2022)
- [Parallel photonic accelerator for decision making using optical spatiotemporal chaos](https://mlsanity.com/paper/2210.06976/) (2022)
- [Decision Maker based on Atomic Switches](https://mlsanity.com/paper/1507.05895/) (2015)
- [Asymmetric leader-laggard cluster synchronization for collective decision-making with laser network](https://mlsanity.com/paper/2312.02537/) (2023)

## Abstract (arXiv)

Hardware accelerators based on physical dynamical systems offer an attractive route toward energy-efficient reinforcement learning applications. However, their scalability is challenging because it requires many statistically independent entropy sources. Here, we introduce a quasi-analog decision-making architecture based on asynchronous Boolean networks (or lattices) implemented on a clockless reconfigurable chip. Each node in the network consists of a single logic element that acts as an autonomous entropy source. This architecture gives rise to distributed Boolean chaos, in which a spatially coupled network generates parallel streams of chaotic Boolean transitions with very low statistical dependence between nodes. We experimentally demonstrate parallel decision-making on a 1024-armed bandit problem, which is beyond the scale of previous hardware implementations, while significantly improving power-law scaling performance. Separately, we scale the proposed entropy source to 5120 parallel channels, yielding an aggregate sample generation rate of 2.14 TS/s. Our solution is implemented on a commercial reconfigurable CMOS chip and offers high integration density and ease of programmability. Our results pave the way for using distributed Boolean chaos as a valuable hardware substrate for large-scale reinforcement learning and for the development of fully integrated, high-throughput decision-making accelerators.