# Hybrid, Unified and Iterative: A Novel Framework for Text-based Person Anomaly Retrieval

arXiv: https://arxiv.org/abs/2511.22470 · HTML: https://mlsanity.com/paper/2511.22470/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 18 references that are in the map.
- First posted on arXiv 27 Nov 2025.

## Builds on (most cited first, 9 of 9)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks](https://mlsanity.com/paper/2208.10442/) (2022)
- [Unified Contrastive Learning in Image-Text-Label Space](https://mlsanity.com/paper/2204.03610/) (2022)
- [Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little](https://mlsanity.com/paper/2104.06644/) (2021)
- [Dual-Path Convolutional Image-Text Embeddings with Instance Loss](https://mlsanity.com/paper/1711.05535/) (2017)
- [Towards Unified Text-based Person Retrieval: A Large-scale Multi-Attribute and Language Search Benchmark](https://mlsanity.com/paper/2306.02898/) (2023)
- [Improving Generalization in Visual Reasoning via Self-Ensemble](https://mlsanity.com/paper/2410.20883/) (2024)
- [Beyond Walking: A Large-Scale Image-Text Benchmark for Text-based Person Anomaly Search](https://mlsanity.com/paper/2411.17776/) (2024)

## Built on by (most cited first, 9 of 9)

- [ITSELF: Attention Guided Fine-Grained Alignment for Vision-Language Retrieval](https://mlsanity.com/paper/2601.01024/) (2026)
- [ESC: Emotional Self-Correction for Reliable Vision-Language Models](https://mlsanity.com/paper/2607.02089/) (2026)
- [FaLCon: Facet-Anchored Retrieval with Late Consensus for Sim2Real Text-Based Person Anomaly Search](https://mlsanity.com/paper/2608.09474/) (2026)
- [Heterogeneous Vision-Language Ensemble with Disagreement-Aware Reranking for Text-Based Person Anomaly Retrieval](https://mlsanity.com/paper/2608.12843/) (2026)
- [MADTempo: An Interactive System for Multi-Event Temporal Video Retrieval with Query Augmentation](https://mlsanity.com/paper/2512.12929/) (2025)
- [Unified Interactive Multimodal Moment Retrieval via Cascaded Embedding-Reranking and Temporal-Aware Score Fusion](https://mlsanity.com/paper/2512.12935/) (2025)
- [LLandMark: A Multi-Agent Framework for Landmark-Aware Multimodal Interactive Video Retrieval](https://mlsanity.com/paper/2603.02888/) (2026)
- [Efficient INT8 Single-Image Super-Resolution via Deployment-Aware Quantization and Teacher-Guided Training](https://mlsanity.com/paper/2604.20291/) (2026)
- [Bridging the Training-Deployment Gap: Gated Encoding and Multi-Scale Refinement for Efficient Quantization-Aware Image Enhancement](https://mlsanity.com/paper/2604.21743/) (2026)

## Abstract (arXiv)

Text-based person anomaly retrieval has emerged as a challenging task, with most existing approaches relying on complex deep-learning techniques. This raises a research question: How can the model be optimized to achieve greater fine-grained features? To address this, we propose a Local-Global Hybrid Perspective (LHP) module integrated with a Vision-Language Model (VLM), designed to explore the effectiveness of incorporating both fine-grained features alongside coarse-grained features. Additionally, we investigate a Unified Image-Text (UIT) model that combines multiple objective loss functions, including Image-Text Contrastive (ITC), Image-Text Matching (ITM), Masked Language Modeling (MLM), and Masked Image Modeling (MIM) loss. Beyond this, we propose a novel iterative ensemble strategy, by combining iteratively instead of using model results simultaneously like other ensemble methods. To take advantage of the superior performance of the LHP model, we introduce a novel feature selection algorithm based on its guidance, which helps improve the model's performance. Extensive experiments demonstrate the effectiveness of our method in achieving state-of-the-art (SOTA) performance on PAB dataset, compared with previous work, with a 9.70% improvement in R@1, 1.77% improvement in R@5, and 1.01% improvement in R@10.