# FMG-Det: Foundation Model Guided Robust Object Detection

arXiv: https://arxiv.org/abs/2505.23726 · HTML: https://mlsanity.com/paper/2505.23726/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 25 references that are in the map.
- First posted on arXiv 29 May 2025.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [Frustratingly Simple Few-Shot Object Detection](https://mlsanity.com/paper/2003.06957/) (2020)
- [Bridging the Gap to Real-World Object-Centric Learning](https://mlsanity.com/paper/2209.14860/) (2022)
- [Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videos](https://mlsanity.com/paper/2205.14065/) (2022)
- [Discovering Objects that Can Move](https://mlsanity.com/paper/2203.10159/) (2022)
- [Towards Noise-resistant Object Detection with Noisy Annotations](https://mlsanity.com/paper/2003.01285/) (2020)
- [Towards Robust Adaptive Object Detection under Noisy Annotations](https://mlsanity.com/paper/2204.02620/) (2022)
- [Robust Object Detection With Inaccurate Bounding Boxes](https://mlsanity.com/paper/2207.09697/) (2022)
- [Training Object Detectors With Noisy Data](https://mlsanity.com/paper/1905.07202/) (2019)
- [Noisy Annotation Refinement for Object Detection](https://mlsanity.com/paper/2110.10456/) (2021)
- [Spatial Self-Distillation for Object Detection with Inaccurate Bounding Boxes](https://mlsanity.com/paper/2307.12101/) (2023)
- [Robust Tiny Object Detection in Aerial Images amidst Label Noise](https://mlsanity.com/paper/2401.08056/) (2024)
- [Distribution-Aware Calibration for Object Detection with Noisy Bounding Boxes](https://mlsanity.com/paper/2308.12017/) (2023)
- [Dynamic Loss Decay based Robust Oriented Object Detection on Remote Sensing Images with Noisy Labels](https://mlsanity.com/paper/2405.09024/) (2024)

## Abstract (arXiv)

Collecting high quality data for object detection tasks is challenging due to the inherent subjectivity in labeling the boundaries of an object. This makes it difficult to not only collect consistent annotations across a dataset but also to validate them, as no two annotators are likely to label the same object using the exact same coordinates. These challenges are further compounded when object boundaries are partially visible or blurred, which can be the case in many domains. Training on noisy annotations significantly degrades detector performance, rendering them unusable, particularly in few-shot settings, where just a few corrupted annotations can impact model performance. In this work, we propose FMG-Det, a simple, efficient methodology for training models with noisy annotations. More specifically, we propose combining a multiple instance learning (MIL) framework with a pre-processing pipeline that leverages powerful foundation models to correct labels prior to training. This pre-processing pipeline, along with slight modifications to the detector head, results in state-of-the-art performance across a number of datasets, for both standard and few-shot scenarios, while being much simpler and more efficient than other approaches.