# ICON: Invariant Counterfactual Optimization with Neuro-Symbolic Priors for Text-Based Person Search

arXiv: https://arxiv.org/abs/2601.15931 · HTML: https://mlsanity.com/paper/2601.15931/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 22 Jan 2026.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)](https://mlsanity.com/paper/1711.09349/) (2017)
- [Joint Detection and Identification Feature Learning for Person Search](https://mlsanity.com/paper/1604.01850/) (2016)
- [Person Re-identification in the Wild](https://mlsanity.com/paper/1604.02531/) (2016)
- [Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval](https://mlsanity.com/paper/2303.12501/) (2023)
- [Anchor-Free Person Search](https://mlsanity.com/paper/2103.11617/) (2021)
- [Sequential End-to-end Network for Efficient Person Search](https://mlsanity.com/paper/2103.10148/) (2021)
- [An Empirical Study of CLIP for Text-based Person Search](https://mlsanity.com/paper/2308.10045/) (2023)
- [PSTR: End-to-End One-Step Person Search With Transformers](https://mlsanity.com/paper/2204.03340/) (2022)
- [Decoupled and Memory-Reinforced Networks: Towards Effective Feature Learning for One-Step Person Search](https://mlsanity.com/paper/2102.10795/) (2021)
- [Text-based Person Search without Parallel Image-Text Data](https://mlsanity.com/paper/2305.12964/) (2023)
- [Text-based Person Search in Full Images via Semantic-Driven Proposal Generation](https://mlsanity.com/paper/2109.12965/) (2021)

## Abstract (arXiv)

Text-Based Person Search (TBPS) holds unique value in real-world surveillance bridging visual perception and language understanding, yet current paradigms utilizing pre-training models often fail to transfer effectively to complex open-world scenarios. The reliance on "Passive Observation" leads to multifaceted spurious correlations and spatial semantic misalignment, causing a lack of robustness against distribution shifts. To fundamentally resolve these defects, this paper proposes ICON (Invariant Counterfactual Optimization with Neuro-symbolic priors), a framework integrating causal and topological priors. First, we introduce Rule-Guided Spatial Intervention to strictly penalize sensitivity to bounding box noise, forcibly severing location shortcuts to achieve geometric invariance. Second, Counterfactual Context Disentanglement is implemented via semantic-driven background transplantation, compelling the model to ignore background interference for environmental independence. Then, we employ Saliency-Driven Semantic Regularization with adaptive masking to resolve local saliency bias and guarantee holistic completeness. Finally, Neuro-Symbolic Topological Alignment utilizes neuro-symbolic priors to constrain feature matching, ensuring activated regions are topologically consistent with human structural logic. Experimental results demonstrate that ICON not only maintains leading performance on standard benchmarks but also exhibits exceptional robustness against occlusion, background interference, and localization noise. This approach effectively advances the field by shifting from fitting statistical co-occurrences to learning causal invariance.