# Do Neural Networks Learn Structure-Preserving Maps? A Case Study in Latent-to-Hilbert Embeddings

arXiv: https://arxiv.org/abs/2610.05297 · HTML: https://mlsanity.com/paper/2610.05297/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 6 references that are in the map.
- First posted on arXiv 4 Oct 2026.

## Builds on (most cited first, 3 of 3)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Supervised learning with quantum enhanced feature spaces](https://mlsanity.com/paper/1804.11326/) (2018)

## Abstract (arXiv)

We ask whether a neural network can learn a structure-preserving map from a compressed latent space to a Hilbert-space representation. Using an 8-dimensional autoencoder bottleneck on MNIST and n-qubit product-state targets from PCA-based angle encoding, we report four findings. Although the target angles are generated by a nonlinear sigmoid transformation of the latent projections, the resulting mapping is well approximated by a linear function over the observed latent distribution: linear regression from z to the true target angles achieves R2 = 0.98, while regression to the MLP's recovered angles achieves R2 = 0.91. The learned map's primary direction is strongly aligned with the target-induced direction, with cosine similarity 0.989, while remaining nearly orthogonal to the input's principal direction, with cosine similarity 0.002. The map is genuinely rank-4: removing any singular direction degrades inner-product preservation by 2.5--5.9× despite a singular-value spectrum with two dominant and two small values. The learned subspace does not coincide with the PCA basis used to construct the target, and different random seeds recover the same primary direction but diverge in higher ranks. Finally, kernel ridge regression with an RBF kernel outperforms a tuned MLP (IP error 0.0144 vs. 0.0197), suggesting that for approximately linear structure-preserving mappings, classical kernel methods may be a simpler and more effective alternative.