# FusionAgent: A Multimodal Agent with Dynamic Model Selection for Human Recognition

arXiv: https://arxiv.org/abs/2603.26908 · HTML: https://mlsanity.com/paper/2603.26908/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 45 of its 70 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 30 of 45)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model](https://mlsanity.com/paper/2504.07615/) (2025)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [Gated Multimodal Units for Information Fusion](https://mlsanity.com/paper/1702.01992/) (2017)
- [Perception-R1: Pioneering Perception Policy with Reinforcement Learning](https://mlsanity.com/paper/2504.07954/) (2025)
- [Clothes-Changing Person Re-identification with RGB Modality Only](https://mlsanity.com/paper/2204.06890/) (2022)
- [Long-Term Cloth-Changing Person Re-identification](https://mlsanity.com/paper/2005.12633/) (2020)
- [MFAS: Multimodal Fusion Architecture Search](https://mlsanity.com/paper/1903.06496/) (2019)
- [Gait Recognition via Disentangled Representation Learning](https://mlsanity.com/paper/1904.04925/) (2019)
- [LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agents](https://mlsanity.com/paper/2503.10200/) (2025)
- [Visual Agentic Reinforcement Fine-Tuning](https://mlsanity.com/paper/2505.14246/) (2025)
- [BigGait: Learning Gait Representation You Want by Large Vision Models](https://mlsanity.com/paper/2402.19122/) (2024)
- [On Learning Disentangled Representations for Gait Recognition](https://mlsanity.com/paper/1909.03051/) (2019)
- [Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector](https://mlsanity.com/paper/2503.20188/) (2025)
- [KeyPoint Relative Position Encoding for Face Recognition](https://mlsanity.com/paper/2403.14852/) (2024)
- [4KAgent: Agentic Any Image to 4K Super-Resolution](https://mlsanity.com/paper/2507.07105/) (2025)
- [Adaptive Graph Representation Learning for Video Person Re-identification](https://mlsanity.com/paper/1909.02240/) (2019)
- [Learning Clothing and Pose Invariant 3D Shape Representation for Long-Term Person Re-Identification](https://mlsanity.com/paper/2308.10658/) (2023)
- [Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMs](https://mlsanity.com/paper/2506.21656/) (2025)
- [FarSight: A Physics-Driven Whole-Body Biometric System at Large Distance and Altitude](https://mlsanity.com/paper/2306.17206/) (2023)
- [TAMM: TriAdapter Multi-Modal Learning for 3D Shape Understanding](https://mlsanity.com/paper/2402.18490/) (2024)
- [STARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Models](https://mlsanity.com/paper/2512.05107/) (2025)
- [Cluster and Aggregate: Face Recognition with Large Probe Set](https://mlsanity.com/paper/2210.10864/) (2022)
- [PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulation](https://mlsanity.com/paper/2601.07060/) (2026)
- [BiggerGait: Unlocking Gait Recognition with Layer-wise Representations from Large Vision Models](https://mlsanity.com/paper/2505.18132/) (2025)

## Built on by (most cited first, 4 of 4)

- [Towards Intrinsic-Aware Monocular 3D Object Detection](https://mlsanity.com/paper/2603.27059/) (2026)
- [DepthAgent: Towards Better Universal Depth Estimation via Sample-wise Expert Selection](https://mlsanity.com/paper/2605.23281/) (2026)
- [Subtoken Vision Transformer for Fine-grained Recognition](https://mlsanity.com/paper/2607.09086/) (2026)
- [Human-Centric Intelligence in the Era of Foundation Models: A Survey](https://mlsanity.com/paper/2608.18184/) (2026)

## Abstract (arXiv)

Model fusion is a key strategy for robust recognition in unconstrained scenarios, as different models provide complementary strengths. This is especially important for whole-body human recognition, where biometric cues such as face, gait, and body shape vary across samples and are typically integrated via score-fusion. However, existing score-fusion strategies are usually static, invoking all models for every test sample regardless of sample quality or modality reliability. To overcome these limitations, we propose FusionAgent, a novel agentic framework that leverages a Multimodal Large Language Model (MLLM) to perform dynamic, sample-specific model selection. Each expert model is treated as a tool, and through Reinforcement Fine-Tuning (RFT) with a metric-based reward, the agent learns to adaptively determine the optimal model combination for each test input. To address the model score misalignment and embedding heterogeneity, we introduce Anchor-based Confidence Top-k (ACT) score-fusion, which anchors on the most confident model and integrates complementary predictions in a confidence-aware manner. Extensive experiments on multiple whole-body biometric benchmarks demonstrate that FusionAgent significantly outperforms SoTA methods while achieving higher efficiency through fewer model invocations, underscoring the critical role of dynamic, explainable, and robust model fusion in real-world recognition systems. Project page: hrefhttps://fusionagent.github.io/FusionAgent.