# Extracting Self-Consistent Causal Insights from Users Feedback with LLMs and In-context Learning

arXiv: https://arxiv.org/abs/2312.06820 · HTML: https://mlsanity.com/paper/2312.06820/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 20 references that are in the map.
- First posted on arXiv 11 Dec 2023.

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [Controlling Linguistic Style Aspects in Neural Language Generation](https://mlsanity.com/paper/1707.02633/) (2017)
- [DoWhy: An End-to-End Library for Causal Inference](https://mlsanity.com/paper/2011.04216/) (2020)
- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [Machine Learning Estimation of Heterogeneous Treatment Effects with Instruments](https://mlsanity.com/paper/1905.10176/) (2019)

## Built on by (most cited first, 5 of 5)

- [Causal Prompting: Debiasing Large Language Model Prompting based on Front-Door Adjustment](https://mlsanity.com/paper/2403.02738/) (2024)
- [Large Language Models and Causal Inference in Collaboration: A Survey](https://mlsanity.com/paper/2403.09606/) (2024)
- [Causality for Large Language Models](https://mlsanity.com/paper/2410.15319/) (2024)
- [CausalEval: Towards Better Causal Reasoning in Language Models](https://mlsanity.com/paper/2410.16676/) (2024)
- [Mitigating Hidden Confounding by Progressive Confounder Imputation via Large Language Models](https://mlsanity.com/paper/2507.02928/) (2025)

## Abstract (arXiv)

Microsoft Windows Feedback Hub is designed to receive customer feedback on a wide variety of subjects including critical topics such as power and battery. Feedback is one of the most effective ways to have a grasp of users' experience with Windows and its ecosystem. However, the sheer volume of feedback received by Feedback Hub makes it immensely challenging to diagnose the actual cause of reported issues. To better understand and triage issues, we leverage Double Machine Learning (DML) to associate users' feedback with telemetry signals. One of the main challenges we face in the DML pipeline is the necessity of domain knowledge for model design (e.g., causal graph), which sometimes is either not available or hard to obtain. In this work, we take advantage of reasoning capabilities in Large Language Models (LLMs) to generate a prior model that which to some extent compensates for the lack of domain knowledge and could be used as a heuristic for measuring feedback informativeness. Our LLM-based approach is able to extract previously known issues, uncover new bugs, and identify sequences of events that lead to a bug, while minimizing out-of-domain outputs.