# Representation-Enhanced Neural Knowledge Integration with Application to Large-Scale Medical Ontology Learning

arXiv: https://arxiv.org/abs/2410.07454 · HTML: https://mlsanity.com/paper/2410.07454/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 9 Oct 2024.

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [A Review of Relational Machine Learning for Knowledge Graphs](https://mlsanity.com/paper/1503.00759/) (2015)
- [Med-HALT: Medical Domain Hallucination Test for Large Language Models](https://mlsanity.com/paper/2307.15343/) (2023)
- [Neural Natural Language Inference Models Enhanced with External Knowledge](https://mlsanity.com/paper/1711.04289/) (2017)
- [Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression](https://mlsanity.com/paper/2210.02002/) (2022)
- [Spectral and matrix factorization methods for consistent community detection in multi-layer networks](https://mlsanity.com/paper/1704.07353/) (2017)
- [Latent space models for multiplex networks with shared structure](https://mlsanity.com/paper/2012.14409/) (2020)
- [Revisiting the Knowledge Injection Frameworks](https://mlsanity.com/paper/2311.01150/) (2023)
- [Statistical Inference on Latent Space Models for Network Data](https://mlsanity.com/paper/2312.06605/) (2023)

## Built on by (most cited first, 4 of 4)

- [LongDiff: Training-Free Long Video Generation in One Go](https://mlsanity.com/paper/2503.18150/) (2025)
- [CEGA: A Cost-Effective Approach for Graph-Based Model Extraction and Acquisition](https://mlsanity.com/paper/2506.17709/) (2025)
- [Knowledge-Embedded Latent Projection for Robust Representation Learning](https://mlsanity.com/paper/2602.16709/) (2026)
- [Toward a Unified Statistical Theory of Unsupervised Pretraining and Supervised Neural Knowledge Graph Learning](https://mlsanity.com/paper/2607.26346/) (2026)

## Abstract (arXiv)

A large-scale knowledge graph enhances reproducibility in biomedical data discovery by providing a standardized, integrated framework that ensures consistent interpretation across diverse datasets. It improves generalizability by connecting data from various sources, enabling broader applicability of findings across different populations and conditions. Generating reliable knowledge graph, leveraging multi-source information from existing literature, however, is challenging especially with a large number of node sizes and heterogeneous relations. In this paper, we propose a general theoretically guaranteed statistical framework, called RENKI, to enable simultaneous learning of multiple relation types. RENKI generalizes various network models widely used in statistics and computer science. The proposed framework incorporates representation learning output into initial entity embedding of a neural network that approximates the score function for the knowledge graph and continuously trains the model to fit observed facts. We prove nonasymptotic bounds for in-sample and out-of-sample weighted MSEs in relation to the pseudo-dimension of the knowledge graph function class. Additionally, we provide pseudo-dimensions for score functions based on multilayer neural networks with ReLU activation function, in the scenarios when the embedding parameters either fixed or trainable. Finally, we complement our theoretical results with numerical studies and apply the method to learn a comprehensive medical knowledge graph combining a pretrained language model representation with knowledge graph links observed in several medical ontologies. The experiments justify our theoretical findings and demonstrate the effect of weighting in the presence of heterogeneous relations and the benefit of incorporating representation learning in nonparametric models.