# Multimodal Object Detection via Probabilistic a priori Information Integration

arXiv: https://arxiv.org/abs/2405.15596 · HTML: https://mlsanity.com/paper/2405.15596/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 24 May 2024.

## Builds on (most cited first, 14 of 14)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images](https://mlsanity.com/paper/1905.12886/) (2019)
- [What Makes Multi-modal Learning Better than Single (Provably)](https://mlsanity.com/paper/2106.04538/) (2021)
- [Class-agnostic Object Detection with Multi-modal Transformer](https://mlsanity.com/paper/2111.11430/) (2021)
- [Toward Transformer-Based Object Detection](https://mlsanity.com/paper/2012.09958/) (2020)
- [AO2-DETR: Arbitrary-Oriented Object Detection Transformer](https://mlsanity.com/paper/2205.12785/) (2022)
- [Deep Lucas-Kanade Homography for Multimodal Image Alignment](https://mlsanity.com/paper/2104.11693/) (2021)
- [Weakly Aligned Feature Fusion for Multimodal Object Detection](https://mlsanity.com/paper/2204.09848/) (2022)
- [Noisy Supervision for Correcting Misaligned Cadaster Maps Without Perfect Ground Truth Data](https://mlsanity.com/paper/1903.06529/) (2019)

## Abstract (arXiv)

Multimodal object detection has shown promise in remote sensing. However, multimodal data frequently encounter the problem of low-quality, wherein the modalities lack strict cell-to-cell alignment, leading to mismatch between different modalities. In this paper, we investigate multimodal object detection where only one modality contains the target object and the others provide crucial contextual information. We propose to resolve the alignment problem by converting the contextual binary information into probability maps. We then propose an early fusion architecture that we validate with extensive experiments on the DOTA dataset.