# Photorealistic Inpainting for Perturbation-based Explanations in Ecological Monitoring

arXiv: https://arxiv.org/abs/2510.03317 · HTML: https://mlsanity.com/paper/2510.03317/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 18 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Builds on (most cited first, 12 of 12)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information](https://mlsanity.com/paper/2402.13616/) (2024)
- [Text-to-Image Models for Counterfactual Explanations: a Black-Box Approach](https://mlsanity.com/paper/2309.07944/) (2023)
- [Rethinking Visual Counterfactual Explanations Through Region Constraint](https://mlsanity.com/paper/2410.12591/) (2024)
- [IBO: Inpainting-Based Occlusion to Enhance Explainable Artificial Intelligence Evaluation in Histopathology](https://mlsanity.com/paper/2408.16395/) (2024)
- [A Meaningful Perturbation Metric for Evaluating Explainability Methods](https://mlsanity.com/paper/2504.06800/) (2025)
- [COIN: Counterfactual inpainting for weakly supervised semantic segmentation for medical images](https://mlsanity.com/paper/2404.12832/) (2024)
- [Inpainting the Gaps: A Novel Framework for Evaluating Explanation Methods in Vision Transformers](https://mlsanity.com/paper/2406.11534/) (2024)

## Built on by (most cited first, 2 of 2)

- [Inpainting Insights: Elevating Visual XAI with Photorealistic Perturbations](https://mlsanity.com/paper/2607.15482/) (2026)
- [Det-LIME: Detector-Aware, Multi-Instance Local Interpretable Model-Agnostic Explanations for Automated Marine Mammal Detection](https://mlsanity.com/paper/2609.17479/) (2026)

## Abstract (arXiv)

Ecological monitoring is increasingly automated by vision models, yet opaque predictions limit trust and field adoption. We present an inpainting-guided, perturbation-based explanation technique that produces photorealistic, mask-localized edits that preserve scene context. Unlike masking or blurring, these edits stay in-distribution and reveal which fine-grained morphological cues drive predictions in tasks such as species recognition and trait attribution. We demonstrate the approach on a YOLOv9 detector fine-tuned for harbor seal detection in Glacier Bay drone imagery, using Segment-Anything-Model-refined masks to support two interventions: (i) object removal/replacement (e.g., replacing seals with plausible ice/water or boats) and (ii) background replacement with original animals composited onto new scenes. Explanations are assessed by re-scoring perturbed images (flip rate, confidence drop) and by expert review for ecological plausibility and interpretability. The resulting explanations localize diagnostic structures, avoid deletion artifacts common to traditional perturbations, and yield domain-relevant insights that support expert validation and more trustworthy deployment of AI in ecology.