# Neuroevolution of Self-Interpretable Agents

arXiv: https://arxiv.org/abs/2003.08165 · HTML: https://mlsanity.com/paper/2003.08165/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2020.

## Built on by (most cited first, 30 of 49)

- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2101.05265/) (2021)
- [Replay-Guided Adversarial Environment Design](https://mlsanity.com/paper/2110.02439/) (2021)
- [A Survey on Interpretable Reinforcement Learning](https://mlsanity.com/paper/2112.13112/) (2021)
- [Automated Reinforcement Learning (AutoRL): A Survey and Open Problems](https://mlsanity.com/paper/2201.03916/) (2022)
- [The State of Sparse Training in Deep Reinforcement Learning](https://mlsanity.com/paper/2206.10369/) (2022)
- [Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robust](https://mlsanity.com/paper/2410.01971/) (2024)
- [Iterative Bounding MDPs: Learning Interpretable Policies via Non-Interpretable Methods](https://mlsanity.com/paper/2102.13045/) (2021)
- [A Survey of Explainable Reinforcement Learning](https://mlsanity.com/paper/2202.08434/) (2022)
- [Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control](https://mlsanity.com/paper/2103.00375/) (2021)
- [Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discovery](https://mlsanity.com/paper/2210.03516/) (2022)
- [Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoires](https://mlsanity.com/paper/2109.08522/) (2021)
- [CLUTR: Curriculum Learning via Unsupervised Task Representation Learning](https://mlsanity.com/paper/2210.10243/) (2022)
- [Stabilizing Unsupervised Environment Design with a Learned Adversary](https://mlsanity.com/paper/2308.10797/) (2023)
- [A Survey of State Representation Learning for Deep Reinforcement Learning](https://mlsanity.com/paper/2506.17518/) (2025)
- [Grounding Aleatoric Uncertainty for Unsupervised Environment Design](https://mlsanity.com/paper/2207.05219/) (2022)
- [Learning to Generalize with Object-centric Agents in the Open World Survival Game Crafter](https://mlsanity.com/paper/2208.03374/) (2022)
- [Learning to Identify Critical States for Reinforcement Learning from Videos](https://mlsanity.com/paper/2308.07795/) (2023)
- [Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examples](https://mlsanity.com/paper/2310.07747/) (2023)
- [Joint Attention for Multi-Agent Coordination and Social Learning](https://mlsanity.com/paper/2104.07750/) (2021)
- [On Generalization Across Environments In Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2503.00799/) (2025)
- [Toward Causal-Aware RL: State-Wise Action-Refined Temporal Difference](https://mlsanity.com/paper/2201.00354/) (2022)
- [Heuristically Adaptive Diffusion-Model Evolutionary Strategy](https://mlsanity.com/paper/2411.13420/) (2024)
- [On learning history based policies for controlling Markov decision processes](https://mlsanity.com/paper/2211.03011/) (2022)
- [Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning](https://mlsanity.com/paper/2310.00229/) (2023)
- [Unlocking Pixels for Reinforcement Learning via Implicit Attention](https://mlsanity.com/paper/2102.04353/) (2021)
- [A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges](https://mlsanity.com/paper/2211.06665/) (2022)
- [Interpretable UAV Collision Avoidance using Deep Reinforcement Learning](https://mlsanity.com/paper/2105.12254/) (2021)
- [Robust Robotic Control from Pixels using Contrastive Recurrent State-Space Models](https://mlsanity.com/paper/2112.01163/) (2021)
- [The Introspective Agent: Interdependence of Strategy, Physiology, and Sensing for Embodied Agents](https://mlsanity.com/paper/2201.00411/) (2022)

## Abstract (arXiv)

Inattentional blindness is the psychological phenomenon that causes one to miss things in plain sight. It is a consequence of the selective attention in perception that lets us remain focused on important parts of our world without distraction from irrelevant details. Motivated by selective attention, we study the properties of artificial agents that perceive the world through the lens of a self-attention bottleneck. By constraining access to only a small fraction of the visual input, we show that their policies are directly interpretable in pixel space. We find neuroevolution ideal for training self-attention architectures for vision-based reinforcement learning (RL) tasks, allowing us to incorporate modules that can include discrete, non-differentiable operations which are useful for our agent. We argue that self-attention has similar properties as indirect encoding, in the sense that large implicit weight matrices are generated from a small number of key-query parameters, thus enabling our agent to solve challenging vision based tasks with at least 1000x fewer parameters than existing methods. Since our agent attends to only task critical visual hints, they are able to generalize to environments where task irrelevant elements are modified while conventional methods fail. Videos of our results and source code available at https://attentionagent.github.io/