# Spectral Analysis of Molecular Features: When Richer Features Do Not Guarantee Better Generalization

arXiv: https://arxiv.org/abs/2510.14217 · HTML: https://mlsanity.com/paper/2510.14217/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 72 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Builds on (most cited first, 30 of 33)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [SchNet: A continuous-filter convolutional neural network for modeling quantum interactions](https://mlsanity.com/paper/1706.08566/) (2017)
- [Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks](https://mlsanity.com/paper/1901.08584/) (2019)
- [ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction](https://mlsanity.com/paper/2010.09885/) (2020)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [Large-Scale Chemical Language Representations Capture Molecular Structure and Properties](https://mlsanity.com/paper/2106.09553/) (2021)
- [The Low-Rank Simplicity Bias in Deep Networks](https://mlsanity.com/paper/2103.10427/) (2021)
- [RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank](https://mlsanity.com/paper/2210.02885/) (2022)
- [Frequency Bias in Neural Networks for Input of Non-Uniform Density](https://mlsanity.com/paper/2003.04560/) (2020)
- [Molecular representation learning with language models and domain-relevant auxiliary tasks](https://mlsanity.com/paper/2011.13230/) (2020)
- [Molecule Attention Transformer](https://mlsanity.com/paper/2002.08264/) (2020)
- [DScribe: Library of Descriptors for Machine Learning in Materials Science](https://mlsanity.com/paper/1904.08875/) (2019)
- [Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language](https://mlsanity.com/paper/2303.03363/) (2023)
- [GAUCHE: A Library for Gaussian Processes in Chemistry](https://mlsanity.com/paper/2212.04450/) (2022)
- [SELFormer: Molecular Representation Learning via SELFIES Language Models](https://mlsanity.com/paper/2304.04662/) (2023)
- [Deep Learning for Computational Chemistry](https://mlsanity.com/paper/1701.04503/) (2017)
- [Generalization in Kernel Regression Under Realistic Assumptions](https://mlsanity.com/paper/2312.15995/) (2023)
- [Domain-Agnostic Molecular Generation with Chemical Feedback](https://mlsanity.com/paper/2301.11259/) (2023)
- [Relative Molecule Self-Attention Transformer](https://mlsanity.com/paper/2110.05841/) (2021)
- [Equivariant Graph Attention Networks for Molecular Property Prediction](https://mlsanity.com/paper/2202.09891/) (2022)
- [TGNN: A Joint Semi-supervised Framework for Graph-level Classification](https://mlsanity.com/paper/2304.11688/) (2023)
- [On the Asymptotic Learning Curves of Kernel Ridge Regression under Power-law Decay](https://mlsanity.com/paper/2309.13337/) (2023)
- [Reliable and Explainable Machine Learning Methods for Accelerated Material Discovery](https://mlsanity.com/paper/1901.02717/) (2019)
- [Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learning](https://mlsanity.com/paper/2508.06199/) (2025)
- [Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum](https://mlsanity.com/paper/2402.01297/) (2024)
- [Kermut: Composite kernel regression for protein variant effects](https://mlsanity.com/paper/2407.00002/) (2024)
- [ADMET property prediction through combinations of molecular fingerprints](https://mlsanity.com/paper/2310.00174/) (2023)
- [A Comprehensive Analysis on the Learning Curve in Kernel Ridge Regression](https://mlsanity.com/paper/2410.17796/) (2024)
- [Impact of Domain Knowledge and Multi-Modality on Intelligent Molecular Property Prediction: A Systematic Survey](https://mlsanity.com/paper/2402.07249/) (2024)

## Abstract (arXiv)

The spectral properties of feature embeddings offer critical insights into model generalization and representation quality. While deep learning models are widely used for molecular property prediction, kernel methods remain competitive in low-data regimes, yet their spectral behavior is largely unexplored. We present the first comprehensive spectral analysis of kernel ridge regression across diverse representations, including molecular fingerprints (ECFP), pretrained transformers, graph neural networks, and 3D descriptors, evaluated on QM9, MoleculeNet, and Therapeutics Data Commons benchmarks. Surprisingly, richer spectral features do not consistently yield better generalization performance, contradicting common representation heuristics used in self-supervised learning (SSL). Across four spectral metrics, only ECFP-based kernels show a strictly positive correlation with performance, particularly SSE, ID, and SR on MoleculeNet. Global 3D representations exhibit mixed behavior; transformer-based representations are consistently positive across all four metrics, whereas local 3D representations show a negative trend and no significant correlations. Truncation analysis further emphasizes this disparity: for local 3D representations on thermodynamic targets, fewer than 2% of eigenvalues, and occasionally as few as 0.02%, are needed to recover 95% of performance, whereas ECFP and transformer kernels require significantly more. By demonstrating a strong dependence on both task and representation, our results challenge the heuristic that richer spectra inherently improve generalization, providing new guidance for evaluating representations in SSL and in label-limited scientific tasks.