mlsanityData to 14 Oct 2026

How Shared Representations Become Specialized: Formation, Maintenance, and Latent Susceptibility

arXiv 2610.15830 ↗First posted 11 Oct 2026cs.LG cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

When several predictions use the same internal representation, information about some targets can remain easier to recover than information about others. We study how this specialization forms, persists, and changes in minimal artificial learners. With reconstruction errors weighted equally during learning, changing which target is observed precisely selects the favored target and redirects an already specialized learner. Removing current support can reduce a formed bias, whereas state-following update feedback can maintain it. We then compare learners formed through different observation and update conditions but matched on specified reconstruction measurements. Both change under the same brief opposing intervention, yet their responses differ. Resetting Adam largely removes the ten-update difference; exchanging its second moment reverses which route is more susceptible. New interventions show that equalizing the mean second-moment scaling reduces but does not remove the difference: its coordinate-wise pattern also matters. With fresh Adam or calibrated SGD, smaller route differences emerge later, whereas 500 common opposing updates bring the selected allocation endpoints close. A further control produces response differences with identical acquisition and equal mean loss weights but jointly different weight variability and realized sequence. Acquisition-selected formation also extends to a symmetric three-target setting. These interventions distinguish specialization's formation, present support, and time-dependent latent susceptibility.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 41 references that are in the map. First posted on arXiv 11 Oct 2026.

Builds on

22 of 41 references.

References · 22

  1. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  2. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  3. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  4. 2015Prioritized Experience Replaycited by 1,099
  5. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  6. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  7. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 496
  8. 2014Recurrent Models of Visual Attentioncited by 349
  9. 2020Performative Predictioncited by 327
  10. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 267
  11. 2017Automated Curriculum Learning for Neural Networkscited by 257
  12. 2017VAMPnets: Deep learning of molecular kineticscited by 136
10 more
  1. 2022Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)cited by 122
  2. 2017How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utilitycited by 120
  3. 2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 67
  4. 2021State Dependent Performative Prediction with Stochastic Approximationcited by 39
  5. 2024Learned feature representations are biased by complexity, learning order, position, and morecited by 23
  6. 2022Active Multi-Task Representation Learningcited by 14
  7. 2024Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasetscited by 6
  8. 2026Stored in Optimizer State, Valued by Later Training: A Causal Account of Subliminal Trait Transfercited by 3
  9. 2026Training Memory in Deep Neural Networks: Mechanisms, Evidence, and Measurement Gapscited by 2
  10. 2026A Minimal Bifurcation Model of Load Imbalance in a Softmax Mixture-of-Experts Routercited by 1

Built on by

Not cited yet.