mlsanityData to 7 Oct 2026

Topological constraints on self-organisation in locally interacting systems

arXiv 2501.13188 ↗First posted 22 Jan 2025cond-mat.stat-mech cs.LG nlin.AOcited by 0

Not on the map. Why?

Abstract (arXiv)

All intelligence is collective intelligence, in the sense that it is made of parts which must align with respect to system-level goals. Understanding the dynamics which facilitate or limit navigation of problem spaces by aligned parts thus impacts many fields ranging across life sciences and engineering. To that end, consider a system on the vertices of a planar graph, with pairwise interactions prescribed by the edges of the graph. Such systems can sometimes exhibit long-range order, distinguishing one phase of macroscopic behaviour from another. In networks of interacting systems we may view spontaneous ordering as a form of self-organisation, modelling neural and basal forms of cognition. Here, we discuss necessary conditions on the topology of the graph for an ordered phase to exist, with an eye towards finding constraints on the ability of a system with local interactions to maintain an ordered target state. By studying the scaling of free energy under the formation of domain walls in three model systems -- the Potts model, autoregressive models, and hierarchical networks -- we show how the combinatorics of interactions on a graph prevent or allow spontaneous ordering. As an application we are able to analyse why multiscale systems like those prevalent in biology are capable of organising into complex patterns, whereas rudimentary language models are challenged by long sequences of outputs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 89 references that are in the map. First posted on arXiv 22 Jan 2025.

Builds on

15 of 89 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  3. 2020Hopfield Networks is All You Needcited by 461
  4. 2021A Survey of Transformerscited by 399
  5. 2016Dense Associative Memory for Pattern Recognitioncited by 230
  6. 2018Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Contextcited by 113
  7. 2022AUC Maximization in the Era of Big Data and AI: A Surveycited by 100
  8. 2021Do Long-Range Language Models Actually Use Long-Range Context?cited by 92
  9. 2024Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Modelscited by 71
  10. 2022Formal Algorithms for Transformerscited by 70
  11. 2022Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Modelscited by 52
  12. 2021Coherence boosting: When your pretrained language model is not paying enough attentioncited by 38
3 more
  1. 2021Technological Approach to Mind Everywhere (TAME): an experimentally-grounded framework for understanding diverse bodies and mindscited by 25
  2. 2023Classical Sorting Algorithms as a Model of Morphogenesis: self-sorting arrays reveal unexpected competencies in a minimal model of basal intelligencecited by 5
  3. 2020Neural and phenotypic representation under the free-energy principlecited by 4

Built on by

Not cited yet.