Can LLMs extract scientific consensus? A case study in high-temperature superconductivity
Not on the map. Why?
Abstract (arXiv)
Scientific knowledge is increasingly dispersed across vast and heterogeneous scientific literature, where important claims are often implicit, evolving, and internally debated. While large language models (LLMs) have shown impressive performance in information extraction and summarization, their ability to recover latent scientific consensus remains unclear. Here, we investigate this problem in the context of high-temperature superconductivity (HTS), a long-standing and highly debated topic in condensed matter physics, as a challenging testbed. Using near 18,000 highly-cited publications over the past seven decades, we construct a structured knowledge graph linking competing superconducting mechanisms, material families, evidential modalities, and citation relations. We find that LLM-extracted representations recover coherent and physically interpretable structures, including family-dependent mechanism profiles, evidence-specific correlations, and citation-mediated temporal evolution of scientific beliefs. Ablation studies on LLM further show that the global structure remains robust across prompting, decoding, and model variations. Our results suggest that LLMs can indeed serve as scalable tools for deciphering scientific knowledge in domains characterized by competing interpretations and evolving knowledge.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 10 of its 39 references that are in the map. First posted on arXiv 26 May 2026.
Builds on
10 of 39 references.
References · 10
- 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
- 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
- 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
- 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 401
- 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
- 2024Language Models with Conformal Factuality Guaranteescited by 128
- 2023Quantifying Uncertainty in Natural Language Explanations of Large Language Modelscited by 31
- 2024LitLLMs, LLMs for Literature Review: Are we there yet?cited by 12
- 2024The Northeast Materials Database for Magnetic Materialscited by 6
- 2025Expert Evaluation of LLM World Models: A High-Tc Superconductivity Case Studycited by 1
Built on by
Not cited yet.