# The Identification and Estimation of Direct and Indirect Effects in A/B Tests through Causal Mediation Analysis

arXiv: https://arxiv.org/abs/1906.09757 · HTML: https://mlsanity.com/paper/1906.09757/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2019.

## Built on by (most cited first, 6 of 6)

- [Causal Mediation Analysis with Hidden Confounders](https://mlsanity.com/paper/2102.11724/) (2021)
- [Disentangled Representation for Causal Mediation Analysis](https://mlsanity.com/paper/2302.09694/) (2023)
- [CETransformer: Casual Effect Estimation via Transformer Based Representation Learning](https://mlsanity.com/paper/2107.08714/) (2021)
- [Towards the D-Optimal Online Experiment Design for Recommender Selection](https://mlsanity.com/paper/2110.12132/) (2021)
- [Multi-output Headed Ensembles for Product Item Classification](https://mlsanity.com/paper/2307.15858/) (2023)
- [Meituan Merchant Business Diagnosis via Policy-Guided Dual-Process User Simulation](https://mlsanity.com/paper/2604.15190/) (2026)

## Abstract (arXiv)

E-commerce companies have a number of online products, such as organic search, sponsored search, and recommendation modules, to fulfill customer needs. Although each of these products provides a unique opportunity for users to interact with a portion of the overall inventory, they are all similar channels for users and compete for limited time and monetary budgets of users. To optimize users' overall experiences on an E-commerce platform, instead of understanding and improving different products separately, it is important to gain insights into the evidence that a change in one product would induce users to change their behaviors in others, which may be due to the fact that these products are functionally similar. In this paper, we introduce causal mediation analysis as a formal statistical tool to reveal the underlying causal mechanisms. Existing literature provides little guidance on cases where multiple unmeasured causally-dependent mediators exist, which are common in A/B tests. We seek a novel approach to identify in those scenarios direct and indirect effects of the treatment. In the end, we demonstrate the effectiveness of the proposed method in data from Etsy's real A/B tests and shed lights on complex relationships between different products.