From Static Bibliometrics to Dynamic Knowledge Graphs: An LLM-Powered Framework for Modernizing Science, Technology, and Innovation (STI) Analytics
Not on the map. Why?
Abstract (arXiv)
Bibliometric indicators - citation counts, h-indexes, co-authorship networks - have long anchored science, technology, and innovation (STI) analytics, yet suffer from temporal lag, semantic shallowness, and an inability to capture the non-linear dynamics of contemporary knowledge ecosystems. Dynamic knowledge graphs and large language models (LLMs) have each been proposed as remedies, but neither is sufficient alone: existing scholarly knowledge graphs remain largely static, while LLM-driven pipelines are prone to hallucination, opacity, and corpus bias without structured grounding. This paper proposes a hybrid, symbolic-first framework integrating all three traditions under explicit methodological constraint. Organized across five layers - an open scholarly data backbone, a dynamic versioned knowledge graph, a constrained LLM-assisted semantic augmentation layer, a multi-layer validation pipeline, and an analytics layer - the framework positions LLMs strictly as generators of provisional candidate enrichments. Candidates become analytically admissible only after passing structural, evidentiary, comparative, and selective expert validation, with full provenance recorded at every stage. The analytics layer supports both established bibliometric indicators and extended graph-based analyses, including trend emergence detection, science-to-technology pathway mapping, and policy-oriented gap analysis. The framework's central theoretical contribution is treating validation as the mediating principle between semantic flexibility and epistemic discipline, enabling STI analytics that is semantically richer and temporally more responsive than static bibliometrics while remaining aligned with the evidentiary standards of science-of-science research. Governance considerations addressing reproducibility, bias, and auditability are also discussed.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 77 references that are in the map. First posted on arXiv 23 Jul 2026.
Builds on
34 of 77 references.
References · 34
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
- 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
- 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
- 2020Knowledge Graphscited by 621
- 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
22 more
- 2018Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Constructioncited by 322
- 2019Entity, Relation, and Event Extraction with Contextualized Span Representationscited by 262
- 2020Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthycited by 183
- 2022OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and conceptscited by 177
- 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunitiescited by 97
- 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
- 2022An Empirical Survey on Long Document Summarization: Datasets, Models and Metricscited by 74
- 2014Growth rates of modern science: A bibliometric analysis based on the number of publications and cited referencescited by 64
- 2022Generative Knowledge Graph Construction: A Reviewcited by 49
- 2024Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspectivecited by 29
- 2025KARMA: Leveraging Multi-Agent LLMs for Automated Knowledge Graph Enrichmentcited by 27
- 2022A Computational Inflection for Scientific Discoverycited by 23
- 2025LLM-empowered knowledge graph construction: A surveycited by 21
- 2020Generating Knowledge Graphs by Employing Natural Language Processing and Machine Learning Techniques within the Scholarly Domaincited by 10
- 2024Reference Coverage Analysis of OpenAlex compared to Web of Science and Scopuscited by 10
- 2025ATOM: AdapTive and OptiMized dynamic temporal knowledge graph construction using LLMscited by 7
- 2023PubGraph: A Large-Scale Scientific Knowledge Graphcited by 6
- 2019Open Research Knowledge Graph: Next Generation Infrastructure for Semantic Scholarly Knowledgecited by 4
- 2024Neurosymbolic Methods for Dynamic Knowledge Graphscited by 3
- 2016Funding acknowledgment analysis:Queries and Caveatscited by 2
- 2024Usage of OpenAlex for creating meaningful global overlay maps of science on the individual and institutional levelscited by 2
- 2025Mitigating LLM Hallucinations with Knowledge Graphs: A Case Studycited by 1
Built on by
Not cited yet.