# Deep forecasting of translational impact in medical research

arXiv: https://arxiv.org/abs/2110.08904 · HTML: https://mlsanity.com/paper/2110.08904/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 59 references that are in the map.
- First posted on arXiv 17 Oct 2021.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [A Web-scale system for scientific knowledge exploration](https://mlsanity.com/paper/1805.12216/) (2018)
- [Hierarchical Block Structures and High-resolution Model Selection in Large Networks](https://mlsanity.com/paper/1310.4377/) (2013)
- [Neural Embeddings of Scholarly Periodicals Reveal Complex Disciplinary Organizations](https://mlsanity.com/paper/2001.08199/) (2020)
- [Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations' COCI: a multidisciplinary comparison of coverage via citations](https://mlsanity.com/paper/2004.14329/) (2020)
- [Understanding the temporal evolution of COVID-19 research through machine learning and natural language processing](https://mlsanity.com/paper/2007.11604/) (2020)
- [How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data](https://mlsanity.com/paper/1612.01822/) (2016)
- [Measuring Social Media Activity of Scientific Literature: An Exhaustive Comparison of Scopus and Novel Altmetrics Big Data](https://mlsanity.com/paper/1705.03258/) (2017)

## Built on by (most cited first, 1 of 1)

- [Machine Learning to Promote Translational Research: Predicting Patent and Clinical Trial Inclusion in Dementia Research](https://mlsanity.com/paper/2401.05145/) (2024)

## Abstract (arXiv)

The value of biomedical research--a 1.7 trillion annual investment--is ultimately determined by its downstream, real-world impact. Current objective predictors of impact rest on proxy, reductive metrics of dissemination, such as paper citation rates, whose relation to real-world translation remains unquantified. Here we sought to determine the comparative predictability of future real-world translation--as indexed by inclusion in patents, guidelines or policy documents--from complex models of the abstract-level content of biomedical publications versus citations and publication meta-data alone. We develop a suite of representational and discriminative mathematical models of multi-scale publication data, quantifying predictive performance out-of-sample, ahead-of-time, across major biomedical domains, using the entire corpus of biomedical research captured by Microsoft Academic Graph from 1990 to 2019, encompassing 43.3 million papers across all domains. We show that citations are only moderately predictive of translational impact as judged by inclusion in patents, guidelines, or policy documents. By contrast, high-dimensional models of publication titles, abstracts and metadata exhibit high fidelity (AUROC > 0.9), generalise across time and thematic domain, and transfer to the task of recognising papers of Nobel Laureates. The translational impact of a paper indexed by inclusion in patents, guidelines, or policy documents can be predicted--out-of-sample and ahead-of-time--with substantially higher fidelity from complex models of its abstract-level content than from models of publication meta-data or citation metrics. We argue that content-based models of impact are superior in performance to conventional, citation-based measures, and sustain a stronger evidence-based claim to the objective measurement of translational potential.