# A Causal Inference Approach for Quantifying Research Impact

arXiv: https://arxiv.org/abs/2503.13485 · HTML: https://mlsanity.com/paper/2503.13485/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 39 references that are in the map.
- First posted on arXiv 7 Mar 2025.

## Builds on (most cited first, 11 of 11)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [A Web-scale system for scientific knowledge exploration](https://mlsanity.com/paper/1805.12216/) (2018)
- [Quantifying Long-Term Scientific Impact](https://mlsanity.com/paper/1306.3293/) (2013)
- [Predicting citation counts based on deep neural network learning techniques](https://mlsanity.com/paper/1809.04365/) (2018)
- [A review of the literature on citation impact indicators](https://mlsanity.com/paper/1507.02099/) (2015)
- [Will This Paper Increase Your h-index? Scientific Impact Prediction](https://mlsanity.com/paper/1412.4754/) (2014)
- [Population-Scale Study of Human Needs During the COVID-19 Pandemic: Analysis and Implications](https://mlsanity.com/paper/2008.07045/) (2020)
- [Online Actions with Offline Impact: How Online Social Networks Influence Online and Offline User Behavior](https://mlsanity.com/paper/1612.03053/) (2016)
- [Should citations be field-normalized in evaluative bibliometrics? An empirical analysis based on propensity score matching](https://mlsanity.com/paper/1910.11706/) (2019)

## Abstract (arXiv)

Deep learning has had a great impact on various fields of computer science by enabling data-driven representation learning in a decade. Because science and technology policy decisions for a nation can be made on the impact of each technology, quantifying research impact is an important task. The number of citations and impact factor can be used to measure the impact for individual research. What would have happened without the research, however, is fundamentally a counterfactual phenomenon. Thus, we propose an approach based on causal inference to quantify the research impact of a specific technical topic. We leverage difference-in-difference to quantify the research impact by applying to bibliometric data. First, we identify papers of a specific technical topic using keywords or category tags from Microsoft Academic Graph, which is one of the largest academic publication dataset. Next, we build a paper citation network between each technical field. Then, we aggregate the cross-field citation count for each research field. Finally, the impact of a specific technical topic for each research field is estimated by applying difference-in-difference. Evaluation results show that deep learning significantly affects computer vision and natural language processing. Besides, deep learning significantly affects cross-field citation especially for speech recognition to computer vision and natural language processing to computer vision. Moreover, our method revealed that the impact of deep learning was 3.1 times of the impact of interpretability for ML models.