# Weakly Supervised Instance Attention for Multisource Fine-Grained Object Recognition with an Application to Tree Species Classification

arXiv: https://arxiv.org/abs/2105.10983 · HTML: https://mlsanity.com/paper/2105.10983/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 45 references that are in the map.
- First posted on arXiv 23 May 2021.

## Builds on (most cited first, 8 of 8)

- [Weakly Supervised Deep Detection Networks](https://mlsanity.com/paper/1511.02853/) (2015)
- [Fine-Grained Object Recognition and Zero-Shot Learning in Remote Sensing Imagery](https://mlsanity.com/paper/1712.03323/) (2017)
- [Recurrently Exploring Class-wise Attention in A Hybrid Convolutional and Bidirectional LSTM Network for Multi-label Aerial Image Classification](https://mlsanity.com/paper/1807.11245/) (2018)
- [Learnable Manifold Alignment (LeMA) : A Semi-supervised Cross-modality Learning Framework for Land Cover and Land Use Classification](https://mlsanity.com/paper/1901.02838/) (2019)
- [From Google Maps to a Fine-Grained Catalog of Street trees](https://mlsanity.com/paper/1910.02675/) (2019)
- [Geocoding of trees from street addresses and street-level images](https://mlsanity.com/paper/2002.01708/) (2020)
- [Semisupervised Manifold Alignment of Multimodal Remote Sensing Images](https://mlsanity.com/paper/2104.07803/) (2021)
- [Multisource Region Attention Network for Fine-Grained Object Recognition in Remote Sensing Imagery](https://mlsanity.com/paper/1901.06403/) (2019)

## Built on by (most cited first, 2 of 2)

- [OpenForest: A data catalogue for machine learning in forest monitoring](https://mlsanity.com/paper/2311.00277/) (2023)
- [Tree-D Fusion: Simulation-Ready Tree Dataset from Single Images with Diffusion Priors](https://mlsanity.com/paper/2407.10330/) (2024)

## Abstract (arXiv)

Multisource image analysis that leverages complementary spectral, spatial, and structural information benefits fine-grained object recognition that aims to classify an object into one of many similar subcategories. However, for multisource tasks that involve relatively small objects, even the smallest registration errors can introduce high uncertainty in the classification process. We approach this problem from a weakly supervised learning perspective in which the input images correspond to larger neighborhoods around the expected object locations where an object with a given class label is present in the neighborhood without any knowledge of its exact location. The proposed method uses a single-source deep instance attention model with parallel branches for joint localization and classification of objects, and extends this model into a multisource setting where a reference source that is assumed to have no location uncertainty is used to aid the fusion of multiple sources in four different levels: probability level, logit level, feature level, and pixel level. We show that all levels of fusion provide higher accuracies compared to the state-of-the-art, with the best performing method of feature-level fusion resulting in 53% accuracy for the recognition of 40 different types of trees, corresponding to an improvement of 5.7% over the best performing baseline when RGB, multispectral, and LiDAR data are used. We also provide an in-depth comparison by evaluating each model at various parameter complexity settings, where the increased model capacity results in a further improvement of 6.3% over the default capacity setting.