# Parameter symmetries determine representational geometry in overparameterized nonlinear networks

arXiv: https://arxiv.org/abs/2609.39078 · HTML: https://mlsanity.com/paper/2609.39078/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 54 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Builds on (most cited first, 3 of 3)

- [Convergent Learning: Do different neural networks learn the same representations?](https://mlsanity.com/paper/1511.07543/) (2015)
- [Equinox: neural networks in JAX via callable PyTrees and filtered transformations](https://mlsanity.com/paper/2111.00254/) (2021)
- [Brain-Model Evaluations Need the NeuroAI Turing Test](https://mlsanity.com/paper/2502.16238/) (2025)

## Abstract (arXiv)

Representations are routinely used across machine learning, psychology, and neuroscience to draw inferences about the computations of biological and artificial systems. Such inferences presume a meaningful link between representational geometry and the computation being performed. For artificial neural networks, however, the extent to which function constrains representation remains unclear. One key obstacle is that these networks admit parameter symmetries: changes in parameterization that preserve function exactly while reshaping representational geometry. Here, we show that a broad class of parameter symmetries acts on representations through just three primitive feature transformations: addition, duplication, and scaling. This feature-level characterization yields a closed-form decomposition of representational geometry into essential and auxiliary components, which makes precise how degeneracy in representational geometry can grow with overparameterization even when function is held fixed. Finally, we show that implementation-level selection rules can resolve this degeneracy, yielding identifiable geometries in which features are weighted according to their contributions to the network's function. Together, our results delineate when representations can support inferences about computation, and when they cannot.