# The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning

arXiv: https://arxiv.org/abs/2109.02869 · HTML: https://mlsanity.com/paper/2109.02869/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Sep 2021.

## Built on by (most cited first, 30 of 50)

- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Generalized Decision Transformer for Offline Hindsight Information Matching](https://mlsanity.com/paper/2111.10364/) (2021)
- [General-Purpose In-Context Learning by Meta-Learning Transformers](https://mlsanity.com/paper/2212.04458/) (2022)
- [A Survey on Transformers in Reinforcement Learning](https://mlsanity.com/paper/2301.03044/) (2023)
- [Discovering Evolution Strategies via Meta-Black-Box Optimization](https://mlsanity.com/paper/2211.11260/) (2022)
- [On Transforming Reinforcement Learning by Transformer: The Development Trajectory](https://mlsanity.com/paper/2212.14164/) (2022)
- [Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimization](https://mlsanity.com/paper/2304.03995/) (2023)
- [StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learning](https://mlsanity.com/paper/2110.06206/) (2021)
- [Policy Architectures for Compositional Generalization in Control](https://mlsanity.com/paper/2203.05960/) (2022)
- [Inducing Point Operator Transformer: A Flexible and Scalable Architecture for Solving PDEs](https://mlsanity.com/paper/2312.10975/) (2023)
- [Learning to Generalize with Object-centric Agents in the Open World Survival Game Crafter](https://mlsanity.com/paper/2208.03374/) (2022)
- [Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARL](https://mlsanity.com/paper/2209.09845/) (2022)
- [Decision Transformer under Random Frame Dropping](https://mlsanity.com/paper/2303.03391/) (2023)
- [Goal-Guided Neural Cellular Automata: Learning to Control Self-Organising Systems](https://mlsanity.com/paper/2205.06806/) (2022)
- [Evolution Transformer: In-Context Evolutionary Optimization](https://mlsanity.com/paper/2403.02985/) (2024)
- [Heuristically Adaptive Diffusion-Model Evolutionary Strategy](https://mlsanity.com/paper/2411.13420/) (2024)
- [Minimal Neural Network Models for Permutation Invariant Agents](https://mlsanity.com/paper/2205.07868/) (2022)
- [Learning multi-modal generative models with permutation-invariant encoders and tighter variational objectives](https://mlsanity.com/paper/2309.00380/) (2023)
- [Structurally Flexible Neural Networks: Evolving the Building Blocks for General Agents](https://mlsanity.com/paper/2404.15193/) (2024)
- [An Evolved Universal Transformer Memory](https://mlsanity.com/paper/2410.13166/) (2024)
- [A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges](https://mlsanity.com/paper/2211.06665/) (2022)
- [Policy Learning for Active Target Tracking over Continuous SE(3) Trajectories](https://mlsanity.com/paper/2212.01498/) (2022)
- [Cooperation Is All You Need](https://mlsanity.com/paper/2305.10449/) (2023)
- [Imagination-Augmented Hierarchical Reinforcement Learning for Safe and Interactive Autonomous Driving in Urban Environments](https://mlsanity.com/paper/2311.10309/) (2023)
- [Traffic Signal Control Using Lightweight Transformers: An Offline-to-Online RL Approach](https://mlsanity.com/paper/2312.07795/) (2023)
- [Shared Modular Recurrence in Contextual MDPs for Universal Morphology Control](https://mlsanity.com/paper/2506.08630/) (2025)
- [Collective Intelligence for 2D Push Manipulations with Mobile Robots](https://mlsanity.com/paper/2211.15136/) (2022)
- [XRL-Bench: A Benchmark for Evaluating and Comparing Explainable Reinforcement Learning Techniques](https://mlsanity.com/paper/2402.12685/) (2024)
- [LayerShuffle: Enhancing Robustness in Vision Transformers by Randomizing Layer Execution Order](https://mlsanity.com/paper/2407.04513/) (2024)
- [Discovering Quality-Diversity Algorithms via Meta-Black-Box Optimization](https://mlsanity.com/paper/2502.02190/) (2025)

## Abstract (arXiv)

In complex systems, we often observe complex global behavior emerge from a collection of agents interacting with each other in their environment, with each individual agent acting only on locally available information, without knowing the full picture. Such systems have inspired development of artificial intelligence algorithms in areas such as swarm optimization and cellular automata. Motivated by the emergence of collective behavior from complex cellular systems, we build systems that feed each sensory input from the environment into distinct, but identical neural networks, each with no fixed relationship with one another. We show that these sensory networks can be trained to integrate information received locally, and through communication via an attention mechanism, can collectively produce a globally coherent policy. Moreover, the system can still perform its task even if the ordering of its inputs is randomly permuted several times during an episode. These permutation invariant systems also display useful robustness and generalization properties that are broadly applicable. Interactive demo and videos of our results: https://attentionneuron.github.io/