Visual Causal Feature Learning
Not on the map. Why?
Abstract (arXiv)
We provide a rigorous definition of the visual cause of a behavior that is broadly applicable to the visually driven behavior in humans, animals, neurons, robots and other perceiving systems. Our framework generalizes standard accounts of causal learning to settings in which the causal variables need to be constructed from micro-variables. We prove the Causal Coarsening Theorem, which allows us to gain causal knowledge from observational data with minimal experimental effort. The theorem provides a connection to standard inference techniques in machine learning that identify features of an image that correlate with, but may not cause, the target behavior. Finally, we propose an active learning scheme to learn a manipulator function that performs optimal manipulations on the image to automatically identify the visual cause of a target behavior. We illustrate our inference and learning algorithms in experiments based on both synthetic and real data.
Key facts. Cited by 59 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Dec 2014.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 11; 2022: 14; 2023: 15; 2024: 6; 2025: 10; 2026: 3.
59 papers.
Sort:
By area:
- 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
- 2022Weakly supervised causal representation learningcited by 140
- 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 133
- 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
- 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 106
- 2021Interventional Video Grounding with Dual Contrastive Learningcited by 94
- 2021Towards Principled Disentanglement for Domain Generalizationcited by 91
- 2021Desiderata for Representation Learning: A Causal Perspectivecited by 76
- 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 66
- 2021Learning latent causal graphs via mixture oraclescited by 55
- 2021Human Trajectory Prediction via Counterfactual Analysiscited by 42
- 2022Causal Effect Identification in Cluster DAGscited by 36
38 more
- 2023Learning nonparametric latent causal graphs with unknown interventionscited by 34
- 2021Compositional Abstraction Error and a Category of Causal Modelscited by 33
- 2023Neural Algorithmic Reasoning with Causal Regularisationcited by 31
- 2022OLIVES Dataset: Ophthalmic Labels for Investigating Visual Eye Semanticscited by 24
- 2024Robust Emotion Recognition in Context Debiasingcited by 24
- 2022Causal Abstraction with Soft Interventionscited by 23
- 2022Image-based Treatment Effect Heterogeneitycited by 20
- 2022Counterfactual Intervention Feature Transfer for Visible-Infrared Person Re-identificationcited by 18
- 2024The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applicationscited by 18
- 2021The Blessings of Unlabeled Background in Untrimmed Videoscited by 16
- 2023Targeted Reduction of Causal Modelscited by 15
- 2023Causal models in string diagramscited by 14
- 2022On the Equivalence of Causal Models: A Category-Theoretic Approachcited by 13
- 2021Towards Unbiased Visual Emotion Recognition via Causal Interventioncited by 11
- 2022Explanatory Paradigms in Neural Networkscited by 11
- 2022Beyond Object Recognition: A New Benchmark towards Object Concept Learningcited by 11
- 2023Toward Falsifying Causal Graphs Using a Permutation-Based Testcited by 11
- 2023Enhancing Adversarial Contrastive Learning via Adversarial Invariant Regularizationcited by 10
- 2023Integrating Earth Observation Data into Causal Inference: Challenges and Opportunitiescited by 8
- 2022Phenomenological Causalitycited by 7
- 2024Robust Training of Federated Models with Extremely Label Deficiencycited by 7
- 2023Adversarial Attacks on Image Classification Models: FGSM and Patch Attacks and their Impactcited by 6
- 2023RobustCLEVR: A Benchmark and Framework for Evaluating Robustness in Object-centric Learningcited by 6
- 2025What is causal about causal models and representations?cited by 6
- 2025Relaxing partition admissibility in Cluster-DAGs: a causal calculus with arbitrary variable clusteringcited by 5
- 2021iReason: Multimodal Commonsense Reasoning using Videos and Natural Language with Interpretabilitycited by 4
- 2021Encoding Causal Macrovariablescited by 4
- 2022Estimating Causal Effects Under Image Confounding Bias with an Application to Poverty in Africacited by 4
- 2023Neuro-Causal Factor Analysiscited by 4
- 2024The Causal Information Bottleneck and Optimal Causal Variable Abstractionscited by 4
- 2025Combining Causal Models for More Accurate Abstractions of Neural Networkscited by 4
- 2025Exploratory Causal Inference in SAEncecited by 4
- 2022Treatment Learning Causal Transformer for Noisy Image Classificationcited by 3
- 2023Adversarial Attacks on Image Classification Models: Analysis and Defensecited by 2
- 2024Causal Imputation for Counterfactual SCMs: Bridging Graphs and Latent Factor Modelscited by 2
- 2024Causality-Driven Audits of Model Robustnesscited by 2
- 2025CauSight: Learning to Supersense for Visual Causal Discoverycited by 2
- 2025Trust Me, I Know the Way: Predictive Uncertainty in the Presence of Shortcut Learningcited by 1
50 most cited of 59 All 59 (JSON)
- 2026Score-Based Learning of Cluster DAGs from Interventions8 Oct 2026
- 2026Bucketing the Good Apples: A Method for Diagnosing and Improving Causal Abstraction4 May 2026
- 2026Bridge: Basis-Driven Causal Inference Marries VFMs for Domain Generalization29 Apr 2026
- 2025CausalFSFG: Rethinking Few-Shot Fine-Grained Visual Categorization from Causal Perspective25 Dec 2025
- 2025CauSight: Learning to Supersense for Visual Causal Discovery1 Dec 2025
- 2025Relaxing partition admissibility in Cluster-DAGs: a causal calculus with arbitrary variable clustering3 Nov 2025
- 2025Exploratory Causal Inference in SAEnce15 Oct 2025
- 2025Learning Nonlinear Causal Reductions to Explain Reinforcement Learning Policies20 Jul 2025
- 2025Active Inference AI Systems for Scientific Discovery26 Jun 2025
- 2025Causal Feature Learning in the Social Sciences17 Mar 2025
- 2025Combining Causal Models for More Accurate Abstractions of Neural Networks14 Mar 2025
- 2025Trust Me, I Know the Way: Predictive Uncertainty in the Presence of Shortcut Learning13 Feb 2025
38 more
- 2025What is causal about causal models and representations?31 Jan 2025
- 2024The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications2 Dec 2024
- 2024Causality-Driven Audits of Model Robustness30 Oct 2024
- 2024The Causal Information Bottleneck and Optimal Causal Variable Abstractions1 Oct 2024
- 2024Robust Emotion Recognition in Context Debiasing9 Mar 2024
- 2024Causal Imputation for Counterfactual SCMs: Bridging Graphs and Latent Factor Models22 Feb 2024
- 2024Robust Training of Federated Models with Extremely Label Deficiency22 Feb 2024
- 2023Adversarial Attacks on Image Classification Models: Analysis and Defense28 Dec 2023
- 2023Targeted Reduction of Causal Models30 Nov 2023
- 2023RobustCLEVR: A Benchmark and Framework for Evaluating Robustness in Object-centric Learning28 Aug 2023
- 2023Measuring the Effect of Causal Disentanglement on the Adversarial Robustness of Neural Network Models21 Aug 2023
- 2023Adversarial Attacks on Image Classification Models: FGSM and Patch Attacks and their Impact5 Jul 2023
- 2023Learning nonparametric latent causal graphs with unknown interventions5 Jun 2023
- 2023Nonparametric Identifiability of Causal Representations from Unknown Interventions1 Jun 2023
- 2023Neuro-Causal Factor Analysis31 May 2023
- 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses18 May 2023
- 2023Toward Falsifying Causal Graphs Using a Permutation-Based Test16 May 2023
- 2023Enhancing Adversarial Contrastive Learning via Adversarial Invariant Regularization30 Apr 2023
- 2023Causal models in string diagrams15 Apr 2023
- 2023Neural Algorithmic Reasoning with Causal Regularisation20 Feb 2023
- 2023Integrating Earth Observation Data into Causal Inference: Challenges and Opportunities30 Jan 2023
- 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability11 Jan 2023
- 2022Beyond Object Recognition: A New Benchmark towards Object Concept Learning6 Dec 2022
- 2022Causal Abstraction with Soft Interventions22 Nov 2022
- 2022Phenomenological Causality15 Nov 2022
- 2022OLIVES Dataset: Ophthalmic Labels for Investigating Visual Eye Semantics22 Sep 2022
- 2022Counterfactual Intervention Feature Transfer for Visible-Infrared Person Re-identification1 Aug 2022
- 2022Attribute Graphs Underlying Molecular Generative Models: Path to Learning with Limited Data14 Jul 2022
- 2022Image-based Treatment Effect Heterogeneity13 Jun 2022
- 2022Estimating Causal Effects Under Image Confounding Bias with an Application to Poverty in Africa13 Jun 2022
- 2022Weakly supervised causal representation learning30 Mar 2022
- 2022Treatment Learning Causal Transformer for Noisy Image Classification29 Mar 2022
- 2022Explanatory Paradigms in Neural Networks24 Feb 2022
- 2022Causal Effect Identification in Cluster DAGs22 Feb 2022
- 2022CITRIS: Causal Identifiability from Temporal Intervened Sequences7 Feb 2022
- 2022On the Equivalence of Causal Models: A Category-Theoretic Approach18 Jan 2022
- 2021Encoding Causal Macrovariables29 Nov 2021
- 2021Towards Principled Disentanglement for Domain Generalization27 Nov 2021
50 newest of 59 All 59 (JSON)
- 2023Neural Algorithmic Reasoning with Causal Regularisationcited by 31
- 2023Causal models in string diagramscited by 14
- 2023Causal models in string diagrams15 Apr 2023
- 2023Neural Algorithmic Reasoning with Causal Regularisation20 Feb 2023
- 2022Weakly supervised causal representation learningcited by 140
- 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 133
- 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
- 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 106
- 2021Interventional Video Grounding with Dual Contrastive Learningcited by 94
- 2021Towards Principled Disentanglement for Domain Generalizationcited by 91
- 2021Desiderata for Representation Learning: A Causal Perspectivecited by 76
- 2023How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defensescited by 66
- 2021Learning latent causal graphs via mixture oraclescited by 55
- 2021Human Trajectory Prediction via Counterfactual Analysiscited by 42
- 2022Causal Effect Identification in Cluster DAGscited by 36
- 2023Learning nonparametric latent causal graphs with unknown interventionscited by 34
12 most cited of 54 All 54 (JSON)
- 2026Score-Based Learning of Cluster DAGs from Interventions8 Oct 2026
- 2026Bucketing the Good Apples: A Method for Diagnosing and Improving Causal Abstraction4 May 2026
- 2026Bridge: Basis-Driven Causal Inference Marries VFMs for Domain Generalization29 Apr 2026
- 2025CausalFSFG: Rethinking Few-Shot Fine-Grained Visual Categorization from Causal Perspective25 Dec 2025
- 2025CauSight: Learning to Supersense for Visual Causal Discovery1 Dec 2025
- 2025Relaxing partition admissibility in Cluster-DAGs: a causal calculus with arbitrary variable clustering3 Nov 2025
- 2025Exploratory Causal Inference in SAEnce15 Oct 2025
- 2025Active Inference AI Systems for Scientific Discovery26 Jun 2025
- 2025Causal Feature Learning in the Social Sciences17 Mar 2025
- 2025Trust Me, I Know the Way: Predictive Uncertainty in the Presence of Shortcut Learning13 Feb 2025
- 2025What is causal about causal models and representations?31 Jan 2025
- 2024The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications2 Dec 2024
12 newest of 54 All 54 (JSON)