mlsanityData to 14 Oct 2026

Spectral Analysis of Molecular Features: When Richer Features Do Not Guarantee Better Generalization

arXiv 2510.14217 ↗First posted 16 Oct 2025cs.LG physics.chem-phcited by 0

Not on the map. Why?

Abstract (arXiv)

The spectral properties of feature embeddings offer critical insights into model generalization and representation quality. While deep learning models are widely used for molecular property prediction, kernel methods remain competitive in low-data regimes, yet their spectral behavior is largely unexplored. We present the first comprehensive spectral analysis of kernel ridge regression across diverse representations, including molecular fingerprints (ECFP), pretrained transformers, graph neural networks, and 3D descriptors, evaluated on QM9, MoleculeNet, and Therapeutics Data Commons benchmarks. Surprisingly, richer spectral features do not consistently yield better generalization performance, contradicting common representation heuristics used in self-supervised learning (SSL). Across four spectral metrics, only ECFP-based kernels show a strictly positive correlation with performance, particularly SSE, ID, and SR on MoleculeNet. Global 3D representations exhibit mixed behavior; transformer-based representations are consistently positive across all four metrics, whereas local 3D representations show a negative trend and no significant correlations. Truncation analysis further emphasizes this disparity: for local 3D representations on thermodynamic targets, fewer than 2% of eigenvalues, and occasionally as few as 0.02%, are needed to recover 95% of performance, whereas ECFP and transformer kernels require significantly more. By demonstrating a strong dependence on both task and representation, our results challenge the heuristic that richer spectra inherently improve generalization, providing new guidance for evaluating representations in SSL and in label-limited scientific tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 72 references that are in the map. First posted on arXiv 16 Oct 2025.

Builds on

33 of 72 references.

References · 33

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  2. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  3. 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 571
  4. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 566
  5. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 337
  6. 2021Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Developmentcited by 264
  7. 2021Large-Scale Chemical Language Representations Capture Molecular Structure and Propertiescited by 206
  8. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 154
  9. 2022RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rankcited by 153
  10. 2020Frequency Bias in Neural Networks for Input of Non-Uniform Densitycited by 140
  11. 2020Molecular representation learning with language models and domain-relevant auxiliary taskscited by 94
  12. 2020Molecule Attention Transformercited by 92
21 more
  1. 2019DScribe: Library of Descriptors for Machine Learning in Materials Sciencecited by 63
  2. 2023Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Languagecited by 44
  3. 2022GAUCHE: A Library for Gaussian Processes in Chemistrycited by 41
  4. 2023SELFormer: Molecular Representation Learning via SELFIES Language Modelscited by 40
  5. 2017Deep Learning for Computational Chemistrycited by 38
  6. 2023Generalization in Kernel Regression Under Realistic Assumptionscited by 27
  7. 2023Domain-Agnostic Molecular Generation with Chemical Feedbackcited by 22
  8. 2021Relative Molecule Self-Attention Transformercited by 21
  9. 2022Equivariant Graph Attention Networks for Molecular Property Predictioncited by 20
  10. 2023TGNN: A Joint Semi-supervised Framework for Graph-level Classificationcited by 19
  11. 2023On the Asymptotic Learning Curves of Kernel Ridge Regression under Power-law Decaycited by 19
  12. 2019Reliable and Explainable Machine Learning Methods for Accelerated Material Discoverycited by 18
  13. 2025Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learningcited by 15
  14. 2024Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrumcited by 13
  15. 2024Kermut: Composite kernel regression for protein variant effectscited by 12
  16. 2023ADMET property prediction through combinations of molecular fingerprintscited by 11
  17. 2024A Comprehensive Analysis on the Learning Curve in Kernel Ridge Regressioncited by 7
  18. 2024Impact of Domain Knowledge and Multi-Modality on Intelligent Molecular Property Prediction: A Systematic Surveycited by 6
  19. 2023Machine Learning Small Molecule Properties in Drug Discoverycited by 5
  20. 2024A Large Encoder-Decoder Family of Foundation Models For Chemical Languagecited by 5
  21. 2022Target alignment in truncated kernel ridge regressioncited by 4

Built on by

Not cited yet.