# What If We Had Used a Different App? Reliable Counterfactual KPI Analysis in Wireless Systems

arXiv: https://arxiv.org/abs/2410.00150 · HTML: https://mlsanity.com/paper/2410.00150/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 69 references that are in the map.
- First posted on arXiv 30 Sep 2024.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review](https://mlsanity.com/paper/2010.10596/) (2020)
- [Conformal Inference of Counterfactuals and Individual Treatment Effects](https://mlsanity.com/paper/2006.06138/) (2020)
- [A comparison of some conformal quantile regression methods](https://mlsanity.com/paper/1909.05433/) (2019)
- [Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications](https://mlsanity.com/paper/2103.04244/) (2021)
- [Understanding O-RAN: Architecture, Interfaces, Algorithms, Security, and Research Challenges](https://mlsanity.com/paper/2202.01032/) (2022)
- [Conformal Prediction Under Feedback Covariate Shift for Biomolecular Design](https://mlsanity.com/paper/2202.03613/) (2022)
- [Federated Conformal Predictors for Distributed Uncertainty Quantification](https://mlsanity.com/paper/2305.17564/) (2023)
- [Conformal Sensitivity Analysis for Individual Treatment Effects](https://mlsanity.com/paper/2112.03493/) (2021)
- [Digital Twin of Wireless Systems: Overview, Taxonomy, Challenges, and Opportunities](https://mlsanity.com/paper/2202.02559/) (2022)
- [Doubly Robust Calibration of Prediction Sets under Covariate Shift](https://mlsanity.com/paper/2203.01761/) (2022)
- [Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)](https://mlsanity.com/paper/2405.06627/) (2024)
- [Conformal Off-Policy Prediction in Contextual Bandits](https://mlsanity.com/paper/2206.04405/) (2022)
- [Forking Uncertainties: Reliable Prediction and Model Predictive Control with Sequence Models via Conformal Risk Control](https://mlsanity.com/paper/2310.10299/) (2023)
- [Conformal Prediction for Causal Effects of Continuous Treatments](https://mlsanity.com/paper/2407.03094/) (2024)
- [5G Wireless Network Slicing for eMBB, URLLC, and mMTC: A Communication-Theoretic View](https://mlsanity.com/paper/1804.05057/) (2018)
- [A Bayesian Framework for Digital Twin-Based Control, Monitoring, and Data Collection in Wireless Systems](https://mlsanity.com/paper/2212.01351/) (2022)
- [dApps: Distributed Applications for Real-time Inference and Control in O-RAN](https://mlsanity.com/paper/2203.02370/) (2022)
- [Causal Reasoning: Charting a Revolutionary Course for Next-Generation AI-Native Wireless Networks](https://mlsanity.com/paper/2309.13223/) (2023)
- [Network Digital Twin: Context, Enabling Technologies and Opportunities](https://mlsanity.com/paper/2205.14206/) (2022)
- [Multi-Source Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2405.09331/) (2024)
- [Federated Inference with Reliable Uncertainty Quantification over Wireless Channels via Conformal Prediction](https://mlsanity.com/paper/2308.04237/) (2023)
- [PACIFISTA: Conflict Evaluation and Management in Open RAN](https://mlsanity.com/paper/2405.04395/) (2024)
- [Guaranteed Dynamic Scheduling of Ultra-Reliable Low-Latency Traffic via Conformal Prediction](https://mlsanity.com/paper/2302.07675/) (2023)
- [Actor-Critic Network for O-RAN Resource Allocation: xApp Design, Deployment, and Analysis](https://mlsanity.com/paper/2210.04604/) (2022)

## Built on by (most cited first, 6 of 6)

- [Conformal Calibration: Ensuring the Reliability of Black-Box AI in Wireless Systems](https://mlsanity.com/paper/2504.09310/) (2025)
- [Ensuring Reliability via Hyperparameter Selection: Review and Advances](https://mlsanity.com/paper/2502.04206/) (2025)
- [Should I Have Expressed a Different Intent? Counterfactual Generation for LLM-Based Autonomous Control](https://mlsanity.com/paper/2601.20090/) (2026)
- [Confounding-Valid Conformal Inference for Counterfactual KPIs in Wireless Networks](https://mlsanity.com/paper/2609.05073/) (2026)
- [Conformal Individual Treatment Effect Estimation under Networked Interference](https://mlsanity.com/paper/2609.15254/) (2026)
- [CALM-5G: Event-Level Conformal Risk Control for Predictive Service-Assurance Alarms](https://mlsanity.com/paper/2610.17962/) (2026)

## Abstract (arXiv)

In modern wireless network architectures, such as Open Radio Access Network (O-RAN), the operation of the radio access network (RAN) is managed by applications, or apps for short, deployed at intelligent controllers. These apps are selected from a given catalog based on current contextual information. For instance, a scheduling app may be selected on the basis of current traffic and network conditions. Once an app is chosen and run, it is no longer possible to directly test the key performance indicators (KPIs) that would have been obtained with another app. In other words, we can never simultaneously observe both the actual KPI, obtained by the selected app, and the counterfactual KPI, which would have been attained with another app, for the same network condition, making individual-level counterfactual KPIs analysis particularly challenging. This what-if analysis, however, would be valuable to monitor and optimize the network operation, e.g., to identify suboptimal app selection strategies. This paper addresses the problem of estimating the values of KPIs that would have been obtained if a different app had been implemented by the RAN. To this end, we propose a conformal-prediction-based counterfactual analysis method for wireless systems that provides reliable error bars for the estimated KPIs, despite the inherent covariate shift between logged and test data. Experimental results for medium access control-layer apps and for physical-layer apps demonstrate the merits of the proposed method.