Semantic Sections: An Atlas-Native Feature Ontology for Obstructed Representation Spaces
Not on the map. Why?
Abstract (arXiv)
Recent interpretability work often treats a feature as a single global direction, dictionary atom, or latent coordinate shared across contexts. We argue that this ontology can fail in obstructed representation spaces, where locally coherent meanings need not assemble into one globally consistent feature. We introduce an atlas-native replacement object, the semantic section: a transport-compatible family of local feature representatives defined over a context atlas. We formalize semantic sections, prove that tree-supported propagation is always pathwise realizable, and show that cycle consistency is the key criterion for genuine globalization. This yields a distinction between tree-local, globalizable, and twisted sections, with twisted sections capturing locally coherent but holonomy-obstructed meanings. We then develop a discovery-and-certification pipeline based on seeded propagation, synchronization across overlaps, defect-based pruning, cycle-aware taxonomy, and deduplication. Across layer-16 atlases for Llama 3.2 3B Instruct, Qwen 2.5 3B Instruct, and Gemma 2 2B IT, we find nontrivial populations of semantic sections, including cycle-supported globalizable and twisted regimes after deduplication. Most importantly, semantic identity is not recovered by raw global-vector similarity. Even certified globalizable sections show low cross-chart signed cosine similarity, and raw similarity baselines recover only a small fraction of true within-section pairs, often collapsing at moderate thresholds. By contrast, section-based identity recovery is perfect on certified supports. These results support semantic sections as a better feature ontology in obstructed regimes.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 10 of its 24 references that are in the map. First posted on arXiv 21 Mar 2026.
Builds on
10 of 24 references.
References · 10
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
- 2022Toy Models of Superpositioncited by 928
- 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
- 2019Intrinsic dimension of data representations in deep neural networkscited by 276
- 2026A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representationscited by 3
Built on by
Not cited yet.