# SupSiam: Non-contrastive Auxiliary Loss for Learning from Molecular Conformers

arXiv: https://arxiv.org/abs/2302.07754 · HTML: https://mlsanity.com/paper/2302.07754/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 29 references that are in the map.
- First posted on arXiv 15 Feb 2023.

## Builds on (most cited first, 14 of 14)

- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields](https://mlsanity.com/paper/2206.07697/) (2022)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs](https://mlsanity.com/paper/2206.11990/) (2022)
- [Pre-training via Denoising for Molecular Property Prediction](https://mlsanity.com/paper/2206.00133/) (2022)
- [GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensembles](https://mlsanity.com/paper/2106.07802/) (2021)
- [Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking](https://mlsanity.com/paper/2111.07786/) (2021)
- [Simple GNN Regularisation for 3D Molecular Property Prediction & Beyond](https://mlsanity.com/paper/2106.07971/) (2021)
- [Understanding Collapse in Non-Contrastive Siamese Representation Learning](https://mlsanity.com/paper/2209.15007/) (2022)
- [Improved Protein-ligand Binding Affinity Prediction with Structure-Based Deep Fusion Inference](https://mlsanity.com/paper/2005.07704/) (2020)
- [Molecular machine learning with conformer ensembles](https://mlsanity.com/paper/2012.08452/) (2020)

## Built on by (most cited first, 3 of 3)

- [On the Effectiveness of Supervision in Asymmetric Non-Contrastive Learning](https://mlsanity.com/paper/2406.10815/) (2024)
- [NEBULA: Neural Empirical Bayes Under Latent Representations for Efficient and Controllable Design of Molecular Libraries](https://mlsanity.com/paper/2407.03428/) (2024)
- [MoleCLUEs: Molecular Conformers Maximally In-Distribution for Predictive Models](https://mlsanity.com/paper/2306.11681/) (2023)

## Abstract (arXiv)

We investigate Siamese networks for learning related embeddings for augmented samples of molecular conformers. We find that a non-contrastive (positive-pair only) auxiliary task aids in supervised training of Euclidean neural networks (E3NNs) and increases manifold smoothness (MS) around point-cloud geometries. We demonstrate this property for multiple drug-activity prediction tasks while maintaining relevant performance metrics, and propose an extension of MS to probabilistic and regression settings. We provide an analysis of representation collapse, finding substantial effects of task-weighting, latent dimension, and regularization. We expect the presented protocol to aid in the development of reliable E3NNs from molecular conformers, even for small-data drug discovery programs.