# Where Animacy Lives in Large Language Models: Tracing the Circuits of the Animacy Concept

arXiv: https://arxiv.org/abs/2607.20995 · HTML: https://mlsanity.com/paper/2607.20995/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 23 Jul 2026.

## Builds on (most cited first, 19 of 19)

- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [How to use and interpret activation patching](https://mlsanity.com/paper/2404.15255/) (2024)
- [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023)
- [When Attention Sink Emerges in Language Models: An Empirical View](https://mlsanity.com/paper/2410.10781/) (2024)
- [The Hydra Effect: Emergent Self-repair in Language Model Computations](https://mlsanity.com/paper/2307.15771/) (2023)
- [Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms](https://mlsanity.com/paper/2403.17806/) (2024)
- [Event knowledge in large language models: the gap between the impossible and the unlikely](https://mlsanity.com/paper/2212.01488/) (2022)
- [Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networks](https://mlsanity.com/paper/2407.04690/) (2024)
- [EAP-GP: Mitigating Saturation Effect in Gradient-based Automated Circuit Identification](https://mlsanity.com/paper/2502.06852/) (2025)
- [Are distributional representations ready for the real world? Evaluating word vectors for grounded perceptual meaning](https://mlsanity.com/paper/1705.11168/) (2017)
- [Living Machines: A study of atypical animacy](https://mlsanity.com/paper/2005.11140/) (2020)
- [When Language Models Fall in Love: Animacy Processing in Transformer Language Models](https://mlsanity.com/paper/2310.15004/) (2023)
- [Are formal and functional linguistic mechanisms dissociated in language models?](https://mlsanity.com/paper/2503.11302/) (2025)

## Abstract (arXiv)

Distinguishing animate from inanimate concepts in written language requires more than shallow text processing, as it involves recognizing complex selectional constraints and contextual cues, such as verb-argument interactions. Yet, current large language models (LLMs) appear to be capable of doing it. We investigate whether this animacy-sensitive behavior of LLMs can be traced to a localized set of causally relevant components and connections. To do so, we construct a controlled dataset of minimal pairs and perform circuit discovery on four open-weight models. Through in-depth experiments and ablations, we show that a causal mechanism responsible for handling animacy in these models does exist, thus discovering an animacy circuit. At the same time, this circuit appears to be less localized compared to other known ones and generalizes only partially across models and animacy tasks, confirming the distributed, context-dependent, and somewhat graded nature of the animacy concept.