mlsanityData to 13 Oct 2026

Artificial Kuramoto Oscillatory Neurons

arXiv 2410.13821 ↗First posted 17 Oct 2024cs.LG cs.AI stat.MLcited by 37

Not on the map. Why?

Abstract (arXiv)

It has long been known in both neuroscience and AI that ``binding'' between neurons leads to a form of competitive learning where representations are compressed in order to represent more abstract concepts in deeper layers of the network. More recently, it was also hypothesized that dynamic (spatiotemporal) representations play an important role in both neuroscience and AI. Building on these ideas, we introduce Artificial Kuramoto Oscillatory Neurons (AKOrN) as a dynamical alternative to threshold units, which can be combined with arbitrary connectivity designs such as fully connected, convolutional, or attentive mechanisms. Our generalized Kuramoto updates bind neurons together through their synchronization dynamics. We show that this idea provides performance improvements across a wide spectrum of tasks such as unsupervised object discovery, adversarial robustness, calibrated uncertainty quantification, and reasoning. We believe that these empirical results show the importance of rethinking our assumptions at the most basic neuronal level of neural representation, and in particular show the importance of dynamical representations. Code:https://github.com/autonomousvision/akorn Project page:https://takerum.github.io/akorn_project_page/

Key facts. Cited by 37 papers in the map (data through 13 Oct 2026). Builds on 74 of its 125 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

74 of 125 references.

References · 74

  1. 2017Attention Is All You Needcited by 60,106
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,550
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,009
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,364
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,336
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,991
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,915
  9. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,379
  10. 2016Layer Normalizationcited by 5,624
  11. 2013Intriguing properties of neural networkscited by 5,137
  12. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,628
38 more
  1. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,539
  2. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,862
  3. 2016Identity Mappings in Deep Residual Networkscited by 1,962
  4. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,612
  5. 2018Group Normalizationcited by 1,518
  6. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,511
  7. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,265
  8. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 919
  9. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 899
  10. 2020Object-Centric Learning with Slot Attentioncited by 869
  11. 2020On Adaptive Attacks to Adversarial Example Defensescited by 554
  12. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 498
  13. 2020Hopfield Networks is All You Needcited by 467
  14. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 403
  15. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 390
  16. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  17. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  18. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 304
  19. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 277
  20. 2021Improving Robustness using Generated Datacited by 234
  21. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 220
  22. 2019SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solvercited by 173
  23. 2020The Hardware Lotterycited by 160
  24. 2021Illiterate DALL-E Learns to Composecited by 147
  25. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  26. 2023The emergence of clusters in self-attention dynamicscited by 135
  27. 2022Graph-Coupled Oscillator Networkscited by 124
  28. 2016Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequencescited by 99
  29. 2021ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentationcited by 93
  30. 2023Object-Centric Slot Diffusioncited by 90
  31. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  32. 2020Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependenciescited by 81
  33. 2023GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformerscited by 77
  34. 2017Recurrent Relational Networkscited by 71
  35. 2021On the effectiveness of adversarial training against common corruptionscited by 71
  36. 2023Energy Transformercited by 70
  37. 2021A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustnesscited by 62
  38. 2022Improving Object-centric Learning with Query Optimizationcited by 60

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 13; 2026: 24.

37 papers.

Sort:

By area:

  1. 2025Continuous Thought Machinescited by 21
  2. 2025Sudoku-Bench: Evaluating creative reasoning with Sudoku variantscited by 21
  3. 2026Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimizationcited by 15
  4. 2025A Formal Comparison Between Chain of Thought and Latent Thoughtcited by 14
  5. 2026Understanding LoRA as Knowledge Memory: An Empirical Analysiscited by 12
  6. 2025Fixed-Point RNNs: Interpolating from Diagonal to Densecited by 5
  7. 2025Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobianscited by 4
  8. 2025Kuramoto Orientation Diffusion Modelscited by 4
  9. 2025Enhancing deep neural networks through complex-valued representations and Kuramoto synchronization dynamicscited by 3
  10. 2025A Circular Argument : Does RoPE need to be Equivariant for Vision?cited by 3
  11. 2026Explore Brain-Inspired Machine Intelligence for Connecting Dots on Graphs Through Holographic Blueprint of Oscillatory Synchronizationcited by 3
  12. 2026Kuramoto Oscillatory Phase Encoding: Neuro-inspired Synchronization for Improved Learning Efficiencycited by 3
25 more
  1. 2026Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectorscited by 3
  2. 2025Hyper-SET: Designing Transformers via Hyperspherical Energy Minimizationcited by 2
  3. 2026Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalizationcited by 2
  4. 2026Spontaneous symmetry breaking and Goldstone modes for deep information propagationcited by 2
  5. 2026Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnosticscited by 2
  6. 2026C-voting: Confidence-Based Test-Time Voting without Explicit Energy Functionscited by 1
  7. 2026Kuramoto Attention: Synchronizing Self-Attention on the Toruscited by 1
  8. 2026Attention as Frustrated Synchronizationcited by 1
  9. 2025Traveling Waves Integrate Spatial Information Through Timecited by 0
  10. 2025Binding threshold units with artificial oscillatory neuronscited by 0
  11. 2025GASPnet: Global Agreement to Synchronize Phasescited by 0
  12. 2025Do traveling waves make good positional encodings?cited by 0
  13. 2026Phasor Agents: Oscillatory Graphs with Three-Factor Plasticity and Sleep-Staged Learningcited by 0
  14. 2026The Phase Is the Gradient: Equilibrium Propagation for Frequency Learning in Kuramoto Networkscited by 0
  15. 2026From Cortical Synchronous Rhythm to Brain Inspired Learning Mechanism: An Oscillatory Spiking Neural Network with Time-Delayed Coordinationcited by 0
  16. 2026Winfree Oscillatory Neural Networkcited by 0
  17. 2026Formalizing the Binding Problemcited by 0
  18. 2026Attention by Synchronization in Coupled Oscillator Networkscited by 0
  19. 2026Graph Coloring Approach to Solving Sudoku with Oscillatory Neural Networkscited by 0
  20. 2026The Kuramoto Neural Operator: Learning to Solve PDEs via Coupled Oscillator Dynamicscited by 0
  21. 2026Position: Certified Correctness in Neural Constraint Reasoning Requires Symbolic Integrationcited by 0
  22. 2026Steering Recurrent Reasoners at Inference Time with Readout Feedbackcited by 0
  23. 2026Neither Adversarial Training Nor Purification: Emergent Adversarial Robustness from Oscillatory Predictive Learningcited by 0
  24. 2026Canonical locks that encode part-whole hierarchiescited by 0
  25. 2026Reasoning with Neural Cellular Automatacited by 0