# How Shared Representations Become Specialized: Formation, Maintenance, and Latent Susceptibility

arXiv: https://arxiv.org/abs/2610.15830 · HTML: https://mlsanity.com/paper/2610.15830/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 11 Oct 2026.

## Builds on (most cited first, 22 of 22)

- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Recurrent Models of Visual Attention](https://mlsanity.com/paper/1406.6247/) (2014)
- [Performative Prediction](https://mlsanity.com/paper/2002.06673/) (2020)
- [Gradient Starvation: A Learning Proclivity in Neural Networks](https://mlsanity.com/paper/2011.09468/) (2020)
- [Automated Curriculum Learning for Neural Networks](https://mlsanity.com/paper/1704.03003/) (2017)
- [VAMPnets: Deep learning of molecular kinetics](https://mlsanity.com/paper/1710.06012/) (2017)
- [Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)](https://mlsanity.com/paper/2203.12221/) (2022)
- [How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility](https://mlsanity.com/paper/1710.11214/) (2017)
- [In Defense of the Unitary Scalarization for Deep Multi-Task Learning](https://mlsanity.com/paper/2201.04122/) (2022)
- [State Dependent Performative Prediction with Stochastic Approximation](https://mlsanity.com/paper/2110.00800/) (2021)
- [Learned feature representations are biased by complexity, learning order, position, and more](https://mlsanity.com/paper/2405.05847/) (2024)
- [Active Multi-Task Representation Learning](https://mlsanity.com/paper/2202.00911/) (2022)
- [Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets](https://mlsanity.com/paper/2410.04579/) (2024)
- [Stored in Optimizer State, Valued by Later Training: A Causal Account of Subliminal Trait Transfer](https://mlsanity.com/paper/2608.20442/) (2026)
- [Training Memory in Deep Neural Networks: Mechanisms, Evidence, and Measurement Gaps](https://mlsanity.com/paper/2601.21624/) (2026)
- [A Minimal Bifurcation Model of Load Imbalance in a Softmax Mixture-of-Experts Router](https://mlsanity.com/paper/2605.29121/) (2026)

## Abstract (arXiv)

When several predictions use the same internal representation, information about some targets can remain easier to recover than information about others. We study how this specialization forms, persists, and changes in minimal artificial learners. With reconstruction errors weighted equally during learning, changing which target is observed precisely selects the favored target and redirects an already specialized learner. Removing current support can reduce a formed bias, whereas state-following update feedback can maintain it. We then compare learners formed through different observation and update conditions but matched on specified reconstruction measurements. Both change under the same brief opposing intervention, yet their responses differ. Resetting Adam largely removes the ten-update difference; exchanging its second moment reverses which route is more susceptible. New interventions show that equalizing the mean second-moment scaling reduces but does not remove the difference: its coordinate-wise pattern also matters. With fresh Adam or calibrated SGD, smaller route differences emerge later, whereas 500 common opposing updates bring the selected allocation endpoints close. A further control produces response differences with identical acquisition and equal mean loss weights but jointly different weight variability and realized sequence. Acquisition-selected formation also extends to a symmetric three-target setting. These interventions distinguish specialization's formation, present support, and time-dependent latent susceptibility.