# Recursive Counterfactual Deconfounding for Object Recognition

arXiv: https://arxiv.org/abs/2309.13924 · HTML: https://mlsanity.com/paper/2309.13924/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 25 Sep 2023.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Open-Set Recognition: a Good Closed-Set Classifier is All You Need?](https://mlsanity.com/paper/2110.06207/) (2021)
- [Causal Intervention for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2009.12547/) (2020)
- [Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1611.02648/) (2016)
- [Reducing Network Agnostophobia](https://mlsanity.com/paper/1811.04110/) (2018)
- [Adversarial Reciprocal Points Learning for Open Set Recognition](https://mlsanity.com/paper/2103.00953/) (2021)
- [C2AE: Class Conditioned Auto-Encoder for Open-set Recognition](https://mlsanity.com/paper/1904.01198/) (2019)
- [OpenGAN: Open-Set Recognition via Open Data Generation](https://mlsanity.com/paper/2104.02939/) (2021)
- [Robust Classification with Convolutional Prototype Learning](https://mlsanity.com/paper/1805.03438/) (2018)
- [Learning Open Set Network with Discriminative Reciprocal Points](https://mlsanity.com/paper/2011.00178/) (2020)
- [Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification](https://mlsanity.com/paper/2108.08728/) (2021)
- [Hybrid Models for Open Set Recognition](https://mlsanity.com/paper/2003.12506/) (2020)
- [Conditional Gaussian Distribution Learning for Open Set Recognition](https://mlsanity.com/paper/2003.08823/) (2020)
- [Counterfactual Zero-Shot and Open-Set Visual Recognition](https://mlsanity.com/paper/2103.00887/) (2021)
- [Human Trajectory Prediction via Counterfactual Analysis](https://mlsanity.com/paper/2107.14202/) (2021)
- [Making Heads or Tails: Towards Semantically Consistent Visual Counterfactuals](https://mlsanity.com/paper/2203.12892/) (2022)
- [ParceLiNGAM: A causal ordering method robust against latent confounders](https://mlsanity.com/paper/1303.7410/) (2013)
- [Class-Specific Semantic Reconstruction for Open Set Recognition](https://mlsanity.com/paper/2207.02158/) (2022)
- [PMAL: Open Set Recognition via Robust Prototype Mining](https://mlsanity.com/paper/2203.08569/) (2022)
- [Difficulty-Aware Simulator for Open Set Recognition](https://mlsanity.com/paper/2207.10024/) (2022)
- [Open-Set Recognition with Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/2006.02003/) (2020)
- [Towards Accurate Open-Set Recognition via Background-Class Regularization](https://mlsanity.com/paper/2207.10287/) (2022)
- [OOD Augmentation May Be at Odds with Open-Set Recognition](https://mlsanity.com/paper/2206.04242/) (2022)
- [A Kernel Embedding-based Approach for Nonstationary Causal Model Inference](https://mlsanity.com/paper/1809.08560/) (2018)

## Abstract (arXiv)

Image recognition is a classic and common task in the computer vision field, which has been widely applied in the past decade. Most existing methods in literature aim to learn discriminative features from labeled images for classification, however, they generally neglect confounders that infiltrate into the learned features, resulting in low performances for discriminating test images. To address this problem, we propose a Recursive Counterfactual Deconfounding model for object recognition in both closed-set and open-set scenarios based on counterfactual analysis, called RCD. The proposed model consists of a factual graph and a counterfactual graph, where the relationships among image features, model predictions, and confounders are built and updated recursively for learning more discriminative features. It performs in a recursive manner so that subtler counterfactual features could be learned and eliminated progressively, and both the discriminability and generalization of the proposed model could be improved accordingly. In addition, a negative correlation constraint is designed for alleviating the negative effects of the counterfactual features further at the model training stage. Extensive experimental results on both closed-set recognition task and open-set recognition task demonstrate that the proposed RCD model performs better than 11 state-of-the-art baselines significantly in most cases.