# HalluciDet: Hallucinating RGB Modality for Person Detection Through Privileged Information

arXiv: https://arxiv.org/abs/2310.04662 · HTML: https://mlsanity.com/paper/2310.04662/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 41 references that are in the map.
- First posted on arXiv 7 Oct 2023.

## Builds on (most cited first, 13 of 13)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [LLVIP: A Visible-infrared Paired Dataset for Low-light Vision](https://mlsanity.com/paper/2108.10831/) (2021)
- [Pyramid Attention Network for Semantic Segmentation](https://mlsanity.com/paper/1805.10180/) (2018)
- [Dark Model Adaptation: Semantic Image Segmentation from Daytime to Nighttime](https://mlsanity.com/paper/1810.02575/) (2018)
- [Dive into Deep Learning](https://mlsanity.com/paper/2106.11342/) (2021)
- [Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blocks](https://mlsanity.com/paper/2009.12664/) (2020)
- [Deep Learning in Robotics: A Review of Recent Research](https://mlsanity.com/paper/1707.07217/) (2017)
- [Distributed Deep Learning Model for Intelligent Video Surveillance Systems with Edge Computing](https://mlsanity.com/paper/1904.06400/) (2019)
- [Deep Learning under Privileged Information Using Heteroscedastic Dropout](https://mlsanity.com/paper/1805.11614/) (2018)
- [Reducing the feature divergence of RGB and near-infrared images using Switchable Normalization](https://mlsanity.com/paper/2106.03088/) (2021)

## Built on by (most cited first, 8 of 8)

- [Visual Modality Prompt for Adapting Vision-Language Object Detectors](https://mlsanity.com/paper/2412.00622/) (2024)
- [MiPa: Mixed Patch Infrared-Visible Modality Agnostic Object Detection](https://mlsanity.com/paper/2404.18849/) (2024)
- [Modality Translation for Object Detection Adaptation Without Forgetting Prior Knowledge](https://mlsanity.com/paper/2404.01492/) (2024)
- [WiSE-OD: Benchmarking Robustness in Infrared Object Detection](https://mlsanity.com/paper/2507.18925/) (2025)
- [MambaTrans: Multimodal Fusion Image Translation via Large Language Model Priors for Downstream Visual Tasks](https://mlsanity.com/paper/2508.07803/) (2025)
- [FreqKD: Frequency-Decoupled Cross-Modal Knowledge Distillation for Infrared Object Detection](https://mlsanity.com/paper/2606.11572/) (2026)
- [InfraNet: Quality-Aware RGB Guidance for Efficient Infrared Object Detection](https://mlsanity.com/paper/2607.03795/) (2026)
- [CFGPNet: Cross-Attention-Based Fused Gradient Programmed Network Framework for Multispectral Object Detection](https://mlsanity.com/paper/2608.06205/) (2026)

## Abstract (arXiv)

A powerful way to adapt a visual recognition model to a new domain is through image translation. However, common image translation approaches only focus on generating data from the same distribution as the target domain. Given a cross-modal application, such as pedestrian detection from aerial images, with a considerable shift in data distribution between infrared (IR) to visible (RGB) images, a translation focused on generation might lead to poor performance as the loss focuses on irrelevant details for the task. In this paper, we propose HalluciDet, an IR-RGB image translation model for object detection. Instead of focusing on reconstructing the original image on the IR modality, it seeks to reduce the detection loss of an RGB detector, and therefore avoids the need to access RGB data. This model produces a new image representation that enhances objects of interest in the scene and greatly improves detection performance. We empirically compare our approach against state-of-the-art methods for image translation and for fine-tuning on IR, and show that our HalluciDet improves detection accuracy in most cases by exploiting the privileged information encoded in a pre-trained RGB detector. Code: https://github.com/heitorrapela/HalluciDet