# BIRD: Distilling Decision Boundaries into Rationales for MLLM Adaptation

arXiv: https://arxiv.org/abs/2609.33713 · HTML: https://mlsanity.com/paper/2609.33713/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 35 of its 39 references that are in the map.
- First posted on arXiv 27 Sep 2026.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 30 of 35)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [STaR: Bootstrapping Reasoning With Reasoning](https://mlsanity.com/paper/2203.14465/) (2022)
- [SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering](https://mlsanity.com/paper/2102.09542/) (2021)
- [SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning](https://mlsanity.com/paper/2408.05517/) (2024)
- [Towards Generalist Biomedical AI](https://mlsanity.com/paper/2307.14334/) (2023)
- [MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding](https://mlsanity.com/paper/2501.18362/) (2025)
- [Multimodal Explanations: Justifying Decisions and Pointing to the Evidence](https://mlsanity.com/paper/1802.08129/) (2018)
- [Improve Vision Language Model Chain-of-thought Reasoning](https://mlsanity.com/paper/2410.16198/) (2024)
- [ChartBench: A Benchmark for Complex Visual Reasoning in Charts](https://mlsanity.com/paper/2312.15915/) (2023)
- [ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering](https://mlsanity.com/paper/2504.05506/) (2025)
- [e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks](https://mlsanity.com/paper/2105.03761/) (2021)
- [Vision-Language Models Can Self-Improve Reasoning via Reflection](https://mlsanity.com/paper/2411.00855/) (2024)
- [ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoning](https://mlsanity.com/paper/2403.09028/) (2024)
- [NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Tasks](https://mlsanity.com/paper/2203.05081/) (2022)
- [Your Vision-Language Model Itself Is a Strong Filter: Towards High-Quality Instruction Tuning with Data Selection](https://mlsanity.com/paper/2402.12501/) (2024)
- [Diving into Self-Evolving Training for Multimodal Reasoning](https://mlsanity.com/paper/2412.17451/) (2024)
- [Chart-based Reasoning: Transferring Capabilities from LLMs to VLMs](https://mlsanity.com/paper/2403.12596/) (2024)
- [HS-STaR: Hierarchical Sampling for Self-Taught Reasoners via Difficulty Estimation and Budget Reallocation](https://mlsanity.com/paper/2505.19866/) (2025)
- [V2T-CoT: From Vision to Text Chain-of-Thought for Medical Reasoning and Diagnosis](https://mlsanity.com/paper/2506.19610/) (2025)
- [Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models](https://mlsanity.com/paper/2407.11422/) (2024)
- [ChartQA-X: Generating Explanations for Visual Chart Reasoning](https://mlsanity.com/paper/2504.13275/) (2025)
- [Through the Lens of Contrast: Self-Improving Visual Reasoning in VLMs](https://mlsanity.com/paper/2603.02556/) (2026)
- [Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs](https://mlsanity.com/paper/2607.03647/) (2026)
- [Efficient End-to-End Visual Document Understanding with Rationale Distillation](https://mlsanity.com/paper/2311.09612/) (2023)
- [Retrieving Counterfactuals Improves Visual In-Context Learning](https://mlsanity.com/paper/2603.16737/) (2026)

## Abstract (arXiv)

Adapting general-purpose multimodal large language models (MLLMs) to specialized domains requires learning domain-specific decision criteria, which often hinge on subtle visual distinctions between otherwise plausible answers. Rationale augmentation aims to expose such evidence through additional observations or inter-sample comparisons, yet a visually valid cue is not necessarily decision-relevant: it may describe how samples differ without changing the model's relative preference between competing answers. We therefore introduce BIRD, a self-improving Boundary-Informed Rationale Distillation framework that uses model-specific confusions to locate unresolved local decision boundaries and distills the evidence that resolves these confusions into rationales. For each sample, BIRD retrieves candidate neighbors from the target MLLM's own representation space and selects the most confusable one according to its answer preferences. It then generates answer-blind candidate evidence from their visual differences and functionally verifies which evidence most effectively strengthens the model's preference for the correct answer while avoiding inappropriate transfer across the pair. The verified evidence is then distilled into a single-sample rationale for standard supervised fine-tuning. Experiments on medical and chart VQA show that BIRD outperforms competing rationale-augmentation methods across two target MLLMs, while further analyses demonstrate clearer separation of confusable answers and stronger gains from model-matched supervision.