# A Cautionary Tale on Integrating Studies with Disparate Outcome Measures for Causal Inference

arXiv: https://arxiv.org/abs/2505.11014 · HTML: https://mlsanity.com/paper/2505.11014/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 16 May 2025.

## Builds on (most cited first, 14 of 14)

- [Discussion on "Bayesian Regression Tree Models for Causal Inference: Regularization, Confounding, and Heterogeneous Effects" by Hahn, Murray and Carvalho](https://mlsanity.com/paper/2108.02836/) (2021)
- [Removing Hidden Confounding by Experimental Grounding](https://mlsanity.com/paper/1810.11646/) (2018)
- [A Review of Generalizability and Transportability](https://mlsanity.com/paper/2102.11904/) (2021)
- [The Influence Function of Semiparametric Estimators](https://mlsanity.com/paper/1508.01378/) (2015)
- [Combining multiple observational data sources to estimate causal effects](https://mlsanity.com/paper/1801.00802/) (2018)
- [Combining Observational and Experimental Datasets Using Shrinkage Estimators](https://mlsanity.com/paper/2002.06708/) (2020)
- [Methods for Integrating Trials and Non-Experimental Data to Examine Treatment Effect Heterogeneity](https://mlsanity.com/paper/2302.13428/) (2023)
- [Combining Experimental and Observational Data for Identification and Estimation of Long-Term Causal Effects](https://mlsanity.com/paper/2201.10743/) (2022)
- [Prognostic Adjustment with Efficient Estimators to Unbiasedly Leverage Historical Data in Randomized Trials](https://mlsanity.com/paper/2305.19180/) (2023)
- [Who Are We Missing? A Principled Approach to Characterizing the Underrepresented Population](https://mlsanity.com/paper/2401.14512/) (2024)
- [Efficient Estimation Under Data Fusion](https://mlsanity.com/paper/2111.14945/) (2021)
- [Towards Generalizing Inferences from Trials to Target Populations](https://mlsanity.com/paper/2402.17042/) (2024)
- [Combining experimental and observational data through a power likelihood](https://mlsanity.com/paper/2304.02339/) (2023)
- [Improving efficiency in transporting average treatment effects](https://mlsanity.com/paper/2304.00117/) (2023)

## Abstract (arXiv)

Data integration approaches are increasingly used to enhance the efficiency and generalizability of studies. However, a key limitation of these methods is the assumption that outcome measures are identical across datasets -- an assumption that often does not hold in practice. Consider the following opioid use disorder (OUD) studies: the XBOT trial and the POAT study, both evaluating the effect of medications for OUD on withdrawal symptom severity (not the primary outcome of either trial). While XBOT measures withdrawal severity using the subjective opiate withdrawal scale, POAT uses the clinical opiate withdrawal scale. We analyze this realistic yet challenging setting where outcome measures differ across studies and where neither study records both types of outcomes. Our paper studies whether and when integrating studies with disparate outcome measures leads to efficiency gains. We introduce three sets of assumptions -- with varying degrees of strength -- linking both outcome measures. Our theoretical and empirical results highlight a cautionary tale: integration can improve asymptotic efficiency only under the strongest assumption linking the outcomes. However, misspecification of this assumption leads to bias. In contrast, a milder assumption may yield finite-sample efficiency gains, yet these benefits diminish as sample size increases. We illustrate these trade-offs via a case study integrating the XBOT and POAT datasets to estimate the comparative effect of two medications for opioid use disorder on withdrawal symptoms. By systematically varying the assumptions linking the SOW and COW scales, we show potential efficiency gains and the risks of bias. Our findings emphasize the need for careful assumption selection when fusing datasets with differing outcome measures, offering guidance for researchers navigating this common challenge in modern data integration.