mlsanityData to 14 Oct 2026

Discovering emergent connections in quantum physics research via dynamic word embeddings

arXiv 2411.06577 ↗First posted 10 Nov 2024cs.LG cs.AI quant-phcited by 3

Not on the map. Why?

Abstract (arXiv)

As the field of quantum physics evolves, researchers naturally form subgroups focusing on specialized problems. While this encourages in-depth exploration, it can limit the exchange of ideas across structurally similar problems in different subfields. To encourage cross-talk among these different specialized areas, data-driven approaches using machine learning have recently shown promise to uncover meaningful connections between research concepts, promoting cross-disciplinary innovation. Current state-of-the-art approaches represent concepts using knowledge graphs and frame the task as a link prediction problem, where connections between concepts are explicitly modeled. In this work, we introduce a novel approach based on dynamic word embeddings for concept combination prediction. Unlike knowledge graphs, our method captures implicit relationships between concepts, can be learned in a fully unsupervised manner, and encodes a broader spectrum of information. We demonstrate that this representation enables accurate predictions about the co-occurrence of concepts within research abstracts over time. To validate the effectiveness of our approach, we provide a comprehensive benchmark against existing methods and offer insights into the interpretability of these embeddings, particularly in the context of quantum physics research. Our findings suggest that this representation offers a more flexible and informative way of modeling conceptual relationships in scientific literature.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 18 of its 42 references that are in the map. First posted on arXiv 10 Nov 2024.

Builds on

18 of 42 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  7. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  8. 2016node2vec: Scalable Feature Learning for Networkscited by 1,918
  9. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
  10. 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Modelscited by 210
  11. 2023SciMON: Scientific Inspiration Machines Optimized for Noveltycited by 179
  12. 2014Temporal Analysis of Language through Neural Language Modelscited by 54
6 more
  1. 2017Dynamic Word Embeddingscited by 51
  2. 2022Predicting the Future of AI with AI: High-quality link prediction in an exponentially growing knowledge networkcited by 36
  3. 2023Accelerating science with human-aware artificial intelligencecited by 28
  4. 2019Predicting Research Trends with Semantic and Neural Networks with an application in Quantum Physicscited by 25
  5. 2024Forecasting high-impact research topics via machine learning on evolving knowledge graphscited by 22
  6. 2018Inorganic Materials Synthesis Planning with Literature-Trained Neural Networkscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.