mlsanityData to 14 Oct 2026

Element2Vec: Build Chemical Element Representation from Text for Property Prediction

arXiv 2510.13916 ↗First posted 15 Oct 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Accurate property data for chemical elements is crucial for materials design and manufacturing, but many of them are difficult to measure directly due to equipment constraints. While traditional methods use the properties of other elements or related properties for prediction via numerical analyses, they often fail to model complex relationships. After all, not all characteristics can be represented as scalars. Recent efforts have been made to explore advanced AI tools such as language models for property estimation, but they still suffer from hallucinations and a lack of interpretability. In this paper, we investigate Element2Vecto effectively represent chemical elements from natural languages to support research in the natural sciences. Given the text parsed from Wikipedia pages, we use language models to generate both a single general-purpose embedding (Global) and a set of attribute-highlighted vectors (Local). Despite the complicated relationship across elements, the computational challenges also exist because of 1) the discrepancy in text distribution between common descriptions and specialized scientific texts, and 2) the extremely limited data, i.e., with only 118 known elements, data for specific properties is often highly sparse and incomplete. Thus, we also design a test-time training method based on self-attention to mitigate the prediction error caused by Vanilla regression clearly. We hope this work could pave the way for advancing AI-driven discovery in materials science.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 62 references that are in the map. First posted on arXiv 15 Oct 2025.

Builds on

28 of 62 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  5. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  6. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  7. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  8. 2023Improving Text Embeddings with Large Language Modelscited by 303
  9. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 302
  10. 2018An efficient framework for learning sentence representationscited by 198
  11. 2023Large Language Models for Generative Information Extraction: A Surveycited by 185
  12. 2022Graph neural networks for materials science and chemistrycited by 182
16 more
  1. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 135
  2. 2025Gemini Embedding: Generalizable Embeddings from Geminicited by 94
  3. 2023Benchmarking large language models for biomedical natural language processing applications and recommendationscited by 78
  4. 2021Partner-Assisted Learning for Few-Shot Image Classificationcited by 36
  5. 2024Complete and Efficient Graph Transformers for Crystal Material Property Predictioncited by 34
  6. 2021Scalable deeper graph neural networks for high-performance materials property predictioncited by 26
  7. 2019CDSA: Cross-Dimensional Self-Attention for Multivariate, Geo-tagged Time Series Imputationcited by 24
  8. 2024MoDE: CLIP Data Experts via Clusteringcited by 22
  9. 2022TempCLR: Temporal Alignment Representation with Contrastive Learningcited by 15
  10. 2024Construction and Application of Materials Knowledge Graph in Multidisciplinary Materials Science via Large Language Modelcited by 15
  11. 2025Periodic Materials Generation using Text-Guided Joint Diffusion Modelcited by 15
  12. 2024Regression with Large Language Models for Materials and Molecular Property Predictioncited by 11
  13. 2021Distributed Representations of Atoms and Materials for Machine Learningcited by 8
  14. 2025Artificial Intelligence and Generative Models for Materials Discovery -- A Reviewcited by 8
  15. 2022Prediction of the electron density of states for crystalline compounds with Atomistic Line Graph Neural Networks (ALIGNN)cited by 6
  16. 2024Way to Specialist: Closing Loop Between Specialized LLM and Evolving Domain Knowledge Graphcited by 6

Built on by

Not cited yet.