mlsanityData to 14 Oct 2026

Unraveling Molecular Structure: A Multimodal Spectroscopic Dataset for Chemistry

arXiv 2407.17492 ↗First posted 4 Jul 2024physics.chem-ph cs.AI cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

Spectroscopic techniques are essential tools for determining the structure of molecules. Different spectroscopic techniques, such as Nuclear magnetic resonance (NMR), Infrared spectroscopy, and Mass Spectrometry, provide insight into the molecular structure, including the presence or absence of functional groups. Chemists leverage the complementary nature of the different methods to their advantage. However, the lack of a comprehensive multimodal dataset, containing spectra from a variety of spectroscopic techniques, has limited machine-learning approaches mostly to single-modality tasks for predicting molecular structures from spectra. Here we introduce a dataset comprising simulated 1H-NMR, 13C-NMR, HSQC-NMR, Infrared, and Mass spectra (positive and negative ion modes) for 790k molecules extracted from chemical reactions in patent data. This dataset enables the development of foundation models for integrating information from multiple spectroscopic modalities, emulating the approach employed by human experts. Additionally, we provide benchmarks for evaluating single-modality tasks such as structure elucidation, predicting the spectra for a target molecule, and functional group predictions. This dataset has the potential automate structure elucidation, streamlining the molecular discovery pipeline from synthesis to structure determination. The dataset and code for the benchmarks can be found at https://rxn4chemistry.github.io/multimodal-spectroscopic-dataset.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 23 of its 75 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

23 of 75 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  5. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  6. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  9. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  10. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  11. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  12. 2016Generative Adversarial Text to Image Synthesiscited by 735
11 more
  1. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 652
  2. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
  3. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  4. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  5. 2018Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Predictioncited by 229
  6. 2017Learning to Plan Chemical Synthesescited by 179
  7. 20234M: Massively Multimodal Masked Modelingcited by 120
  8. 2022Accelerating Material Design with the Generative Toolkit for Scientific Discoverycited by 16
  9. 2023Prefix-Tree Decoding for Predicting Mass Spectra from Moleculescited by 16
  10. 2014Fragmentation trees reloadedcited by 9
  11. 2021Unassisted Noise Reduction of Chemical Reaction Data Setscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 16; 2026: 14.

31 papers.

Sort:

By area:

  1. 2024From Generalist to Specialist: A Survey of Large Language Models for Chemistrycited by 11
  2. 2025NMR-Solver: Automated Structure Elucidation via Large-Scale Spectral Matching and Physics-Guided Fragment Optimizationcited by 11
  3. 2025MolSpectra: Pre-training 3D Molecular Representation with Multi-modal Energy Spectracited by 8
  4. 2025Artificial Intelligence in Spectroscopy: Advancing Chemistry from Prediction to Generation and Beyondcited by 6
  5. 2025Quantum Doubly Stochastic Transformerscited by 6
  6. 2025DiffSpectra: Molecular Structure Elucidation from Spectra using Diffusion Modelscited by 5
  7. 2025DiffNMR: Diffusion Models for Nuclear Magnetic Resonance Spectra Elucidationcited by 5
  8. 2025IR-Agent: Expert-Inspired LLM Agents for Structure Elucidation from Infrared Spectracited by 5
  9. 2025Test-Time Tuned Language Models Enable End-to-end De Novo Molecular Structure Generation from MS/MS Spectracited by 4
  10. 2025Atomic Diffusion Models for Small Molecule Structure Elucidation from NMR Spectracited by 4
  11. 2026FRIGID: Scaling Diffusion-Based Molecular Generation from Mass Spectra at Training and Inference Timecited by 4
  12. 2025SpectraLLM: Uncovering the Ability of LLMs for Molecular Structure Elucidation from Multi-Spectral Datacited by 3
19 more
  1. 2026MassSpecGym in the Wild: Uncovering and Correcting Evaluation Pitfalls in AI-Driven Molecule Discoverycited by 3
  2. 2025SpectrumWorld: Artificial Intelligence Foundation for Spectroscopycited by 2
  3. 2025We Need Improved Data Curation and Attribution in AI for Scientific Discoverycited by 1
  4. 2025Large Language Models Transform Organic Synthesis From Reaction Prediction to Automationcited by 1
  5. 2025CITE: A Comprehensive Benchmark for Heterogeneous Text-Attributed Graphs on Catalytic Materialscited by 1
  6. 2025SpecMol: A Spectroscopy-Grounded Foundation Model for Multi-Task Molecular Learningcited by 1
  7. 2025NMIRacle: Multi-modal Generative Molecular Elucidation from IR and NMR Spectracited by 1
  8. 2026NMRTrans: Structure Elucidation from Experimental NMR Spectra via Set Transformerscited by 1
  9. 2026SpecX: A Large-Scale Benchmark for Multi-Modal Spectroscopy and Cross-Paradigm Evaluationcited by 1
  10. 2026Latent Diffusion-Based 3D Molecular Recovery from Vibrational Spectracited by 0
  11. 2026Molecules Meet Language: Confound-Aware Representation Learning and Chemical Property Steering in Transformer-VAE Latent Spacescited by 0
  12. 2026CoRe-Gen: Robust Spectrum-to-Structure Generation under Imperfect Fingerprint Conditionscited by 0
  13. 2026Hypothesis-and-Refinement Learning of Organic Structures from Multimodal Spectroscopic Datacited by 0
  14. 2026Data Fusion and Contrastive Alignment for Unconstrained IR Molecular Structure Elucidationcited by 0
  15. 2026Simulation-to-real transfer learning for infrared spectroscopic chemical sensing and analysis from molecules to complex samplescited by 0
  16. 2026Multi-Agent Closed-Loop Reasoning for Organic Structure Elucidation from Multimodal Spectracited by 0
  17. 2026Towards Reasonable Molecular Structure Elucidation from Infrared Spectroscopy with Chemical Feedbackcited by 0
  18. 2026S3C-LLM: Skill-Code Guided Agentic Language Models for Spectrum-to-Structure Elucidationcited by 0
  19. 2026Robust small-molecule identification from incomplete, degraded, and inconsistent spectra using multimodal mixed-condition trainingcited by 0