mlsanityData to 14 Oct 2026

SIGMA: Semantic Identifier Grouping for Molecular Autoregression

arXiv 2603.25062 ↗First posted 26 Mar 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Autoregressive molecular models assign probability to molecular serializations even though chemical identity is invariant to serialization. Equivalent serializations can therefore represent a common molecular identity yet induce inconsistent next-token decisions. Randomized strings broaden exposure, but do not reveal which intermediate decisions should agree. We introduce SIGMA, a dense suffix-position objective built from chemically certified same-suffix triplets: two equivalent histories, one non-equivalent history, and a shared suffix. SIGMA aligns corresponding pre-token hidden states along the continuation and separates the negative to a finite relative margin, leaving the language-model objective, decoder, and inference procedure unchanged. We compare SIGMA with canonical training, randomized-serialization training, and last-token alignment across four datasets under SMILES and SELFIES. Across the eight representation-dataset blocks, SIGMA yields clear test-reference Frechet ChemNet Distance reductions in six: all four SELFIES domains and QM9 and ZINC under SMILES, with paired 95% confidence intervals below zero against every control. Position-wise analyses show improved state correspondence, chemical discrimination, and next-token agreement while preserving between-molecule information. On two full-corpus ZINC blocks, compute-matched ablations identify chemically correct state correspondence as the effective ingredient. Beyond generation, SIGMA improves mean predictive performance on all six molecular property benchmarks and reduces sensitivity to equivalent molecular serializations on every task.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 52 references that are in the map. First posted on arXiv 26 Mar 2026.

Builds on

25 of 52 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  6. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  7. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
  8. 2018Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generationcited by 364
  9. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 337
  10. 2020GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generationcited by 305
  11. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 297
  12. 2018Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Modelscited by 275
13 more
  1. 2017Molecular De Novo Design through Deep Reinforcement Learningcited by 267
  2. 2018Fr'echet ChemNet Distance: A metric for generative models for molecules in drug discoverycited by 194
  3. 2022Sample Efficiency Matters: A Benchmark for Practical Molecular Optimizationcited by 155
  4. 2020Molecular representation learning with language models and domain-relevant auxiliary taskscited by 94
  5. 2019SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discoverycited by 90
  6. 2017SMILES Enumeration as Data Augmentation for Neural Network Modeling of Moleculescited by 59
  7. 2023Gotta be SAFE: A New Framework for Molecular Designcited by 48
  8. 2021TaCL: Improving BERT Pre-training with Token-aware Contrastive Learningcited by 32
  9. 2025Learning-Order Autoregressive Models with Application to Molecular Graph Generationcited by 29
  10. 2024Molecule Generation with Fragment Retrieval Augmentationcited by 23
  11. 2023Token-Level Contrastive Learning with Modality-Aware Prompting for Multimodal Intent Recognitioncited by 18
  12. 2023t-SMILES: A Scalable Fragment-based Molecular Representation Framework for De Novo Molecule Generationcited by 14
  13. 2025Leveraging Partial SMILES Validation Scheme for Enhanced Drug Design in Reinforcement Learning Frameworkscited by 3

Built on by

Not cited yet.