# Artificial Kuramoto Oscillatory Neurons

arXiv: https://arxiv.org/abs/2410.13821 · HTML: https://mlsanity.com/paper/2410.13821/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 74 of its 125 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 30 of 74)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Faith and Fate: Limits of Transformers on Compositionality](https://mlsanity.com/paper/2305.18654/) (2023)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples](https://mlsanity.com/paper/1711.09325/) (2017)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [Your Diffusion Model is Secretly a Zero-Shot Classifier](https://mlsanity.com/paper/2303.16203/) (2023)

## Built on by (most cited first, 30 of 38)

- [Continuous Thought Machines](https://mlsanity.com/paper/2505.05522/) (2025)
- [Sudoku-Bench: Evaluating creative reasoning with Sudoku variants](https://mlsanity.com/paper/2505.16135/) (2025)
- [Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimization](https://mlsanity.com/paper/2606.16899/) (2026)
- [A Formal Comparison Between Chain of Thought and Latent Thought](https://mlsanity.com/paper/2509.25239/) (2025)
- [Understanding LoRA as Knowledge Memory: An Empirical Analysis](https://mlsanity.com/paper/2603.01097/) (2026)
- [Fixed-Point RNNs: Interpolating from Diagonal to Dense](https://mlsanity.com/paper/2503.10799/) (2025)
- [Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobians](https://mlsanity.com/paper/2505.19458/) (2025)
- [Kuramoto Orientation Diffusion Models](https://mlsanity.com/paper/2509.15328/) (2025)
- [Enhancing deep neural networks through complex-valued representations and Kuramoto synchronization dynamics](https://mlsanity.com/paper/2502.21077/) (2025)
- [A Circular Argument : Does RoPE need to be Equivariant for Vision?](https://mlsanity.com/paper/2511.08368/) (2025)
- [Explore Brain-Inspired Machine Intelligence for Connecting Dots on Graphs Through Holographic Blueprint of Oscillatory Synchronization](https://mlsanity.com/paper/2602.00057/) (2026)
- [Kuramoto Oscillatory Phase Encoding: Neuro-inspired Synchronization for Improved Learning Efficiency](https://mlsanity.com/paper/2604.07904/) (2026)
- [Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors](https://mlsanity.com/paper/2606.25971/) (2026)
- [Hyper-SET: Designing Transformers via Hyperspherical Energy Minimization](https://mlsanity.com/paper/2502.11646/) (2025)
- [Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization](https://mlsanity.com/paper/2604.01577/) (2026)
- [Spontaneous symmetry breaking and Goldstone modes for deep information propagation](https://mlsanity.com/paper/2605.14685/) (2026)
- [Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnostics](https://mlsanity.com/paper/2605.20441/) (2026)
- [C-voting: Confidence-Based Test-Time Voting without Explicit Energy Functions](https://mlsanity.com/paper/2604.13521/) (2026)
- [Winfree Oscillatory Neural Network](https://mlsanity.com/paper/2605.20922/) (2026)
- [Kuramoto Attention: Synchronizing Self-Attention on the Torus](https://mlsanity.com/paper/2606.11585/) (2026)
- [Attention as Frustrated Synchronization](https://mlsanity.com/paper/2606.18694/) (2026)
- [The Kuramoto Neural Operator: Learning to Solve PDEs via Coupled Oscillator Dynamics](https://mlsanity.com/paper/2608.10234/) (2026)
- [Steering Recurrent Reasoners at Inference Time with Readout Feedback](https://mlsanity.com/paper/2608.24136/) (2026)
- [Traveling Waves Integrate Spatial Information Through Time](https://mlsanity.com/paper/2502.06034/) (2025)
- [Binding threshold units with artificial oscillatory neurons](https://mlsanity.com/paper/2505.03648/) (2025)
- [GASPnet: Global Agreement to Synchronize Phases](https://mlsanity.com/paper/2507.16674/) (2025)
- [Do traveling waves make good positional encodings?](https://mlsanity.com/paper/2511.11668/) (2025)
- [Phasor Agents: Oscillatory Graphs with Three-Factor Plasticity and Sleep-Staged Learning](https://mlsanity.com/paper/2601.04362/) (2026)
- [The Phase Is the Gradient: Equilibrium Propagation for Frequency Learning in Kuramoto Networks](https://mlsanity.com/paper/2604.10272/) (2026)
- [From Cortical Synchronous Rhythm to Brain Inspired Learning Mechanism: An Oscillatory Spiking Neural Network with Time-Delayed Coordination](https://mlsanity.com/paper/2605.01656/) (2026)

## Abstract (arXiv)

It has long been known in both neuroscience and AI that ``binding'' between neurons leads to a form of competitive learning where representations are compressed in order to represent more abstract concepts in deeper layers of the network. More recently, it was also hypothesized that dynamic (spatiotemporal) representations play an important role in both neuroscience and AI. Building on these ideas, we introduce Artificial Kuramoto Oscillatory Neurons (AKOrN) as a dynamical alternative to threshold units, which can be combined with arbitrary connectivity designs such as fully connected, convolutional, or attentive mechanisms. Our generalized Kuramoto updates bind neurons together through their synchronization dynamics. We show that this idea provides performance improvements across a wide spectrum of tasks such as unsupervised object discovery, adversarial robustness, calibrated uncertainty quantification, and reasoning. We believe that these empirical results show the importance of rethinking our assumptions at the most basic neuronal level of neural representation, and in particular show the importance of dynamical representations. Code:https://github.com/autonomousvision/akorn Project page:https://takerum.github.io/akorn_project_page/