# MAT-Agent: Adaptive Multi-Agent Training Optimization

arXiv: https://arxiv.org/abs/2510.17845 · HTML: https://mlsanity.com/paper/2510.17845/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 66 references that are in the map.
- First posted on arXiv 10 Oct 2025.

## Builds on (most cited first, 30 of 45)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)
- [Reinforcement Learning with Unsupervised Auxiliary Tasks](https://mlsanity.com/paper/1611.05397/) (2016)
- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Population Based Training of Neural Networks](https://mlsanity.com/paper/1711.09846/) (2017)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [Fast AutoAugment](https://mlsanity.com/paper/1905.00397/) (2019)
- [Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents](https://mlsanity.com/paper/1802.08757/) (2018)
- [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016)
- [AutoML-Zero: Evolving Machine Learning Algorithms From Scratch](https://mlsanity.com/paper/2003.03384/) (2020)

## Built on by (most cited first, 30 of 35)

- [FastVMT: Eliminating Redundancy in Video Motion Transfer](https://mlsanity.com/paper/2602.05551/) (2026)
- [Top-Down Semantic Refinement for Image Captioning](https://mlsanity.com/paper/2510.22391/) (2025)
- [RaCoT: Plug-and-Play Contrastive Example Generation Mechanism for Enhanced LLM Reasoning Reliability](https://mlsanity.com/paper/2510.22710/) (2025)
- [Cost-Effective Communication: An Auction-based Method for Language Agent Interaction](https://mlsanity.com/paper/2511.13193/) (2025)
- [3DAlign-DAER: Dynamic Attention Policy and Efficient Retrieval Strategy for Fine-grained 3D-Text Alignment at Scale](https://mlsanity.com/paper/2511.13211/) (2025)
- [FlashVLM: Text-Guided Visual Token Selection for Large Multimodal Models](https://mlsanity.com/paper/2512.20561/) (2025)
- [Dual Latent Memory for Visual Multi-agent System](https://mlsanity.com/paper/2602.00471/) (2026)
- [STORM: Search-Guided Generative World Models for Robotic Manipulation](https://mlsanity.com/paper/2512.18477/) (2025)
- [CoAgent: Collaborative Planning and Consistency Agent for Coherent Video Generation](https://mlsanity.com/paper/2512.22536/) (2025)
- [MultiMotion: Multi Subject Video Motion Transfer via Video Diffusion Transformer](https://mlsanity.com/paper/2512.07500/) (2025)
- [Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agents](https://mlsanity.com/paper/2512.08870/) (2025)
- [Large Language Models as Discounted Bayesian Filters](https://mlsanity.com/paper/2512.18489/) (2025)
- [Adaptive-VoCo: Complexity-Aware Visual Token Compression for Vision-Language Models](https://mlsanity.com/paper/2512.18496/) (2025)
- [ResWM: Residual-Action World Model for Visual RL](https://mlsanity.com/paper/2603.11110/) (2026)
- [Backward-Friendly Optimization: Training Large Language Models with Approximate Gradients under Memory Constraints](https://mlsanity.com/paper/2510.22467/) (2025)
- [Agent-GSPO: Communication-Efficient Multi-Agent Systems via Group Sequence Policy Optimization](https://mlsanity.com/paper/2510.22477/) (2025)
- [Reflective Confidence: Correcting Reasoning Flaws via Online Self-Correction](https://mlsanity.com/paper/2512.18605/) (2025)
- [ForgetMark: Stealthy Fingerprint Embedding via Targeted Unlearning in Language Models](https://mlsanity.com/paper/2601.08189/) (2026)
- [A Heuristic Perspective on Debiasing Language Models](https://mlsanity.com/paper/2608.00622/) (2026)
- [Token-Budget Distillation: Transferring Full-Token Semantics to Compressed Video Vision-Language Models](https://mlsanity.com/paper/2608.28138/) (2026)
- [DynaPose4D: High-Quality 4D Dynamic Content Generation via Pose Alignment Loss](https://mlsanity.com/paper/2510.22473/) (2025)
- [GTMA: Dynamic Representation Optimization for OOD Vision-Language Models](https://mlsanity.com/paper/2512.18504/) (2025)
- [PTTA: A Pure Text-to-Animation Framework for High-Quality Creation](https://mlsanity.com/paper/2512.18614/) (2025)
- [SirenPose: Dynamic Scene Reconstruction via Geometric Supervision](https://mlsanity.com/paper/2512.20531/) (2025)
- [RevFFN: Memory-Efficient Full-Parameter Fine-Tuning of Mixture-of-Experts LLMs with Reversible Blocks](https://mlsanity.com/paper/2512.20920/) (2025)
- [A Three-Level Alignment Framework for Large-Scale 3D Retrieval and Controlled 4D Generation](https://mlsanity.com/paper/2512.22294/) (2025)
- [Learning Dynamic Scene Reconstruction with Sinusoidal Geometric Priors](https://mlsanity.com/paper/2512.22295/) (2025)
- [Towards Efficient Post-Training via Fourier-Driven Adapter Architectures](https://mlsanity.com/paper/2512.22378/) (2025)
- [Forgetting as a Feature: Cognitive Alignment of Large Language Models](https://mlsanity.com/paper/2601.09726/) (2025)
- [LogicGaze: Benchmarking Causal Consistency in Visual Narratives via Counterfactual Verification](https://mlsanity.com/paper/2602.00292/) (2026)

## Abstract (arXiv)

Multi-label image classification demands adaptive training strategies to navigate complex, evolving visual-semantic landscapes, yet conventional methods rely on static configurations that falter in dynamic settings. We propose MAT-Agent, a novel multi-agent framework that reimagines training as a collaborative, real-time optimization process. By deploying autonomous agents to dynamically tune data augmentation, optimizers, learning rates, and loss functions, MAT-Agent leverages non-stationary multi-armed bandit algorithms to balance exploration and exploitation, guided by a composite reward harmonizing accuracy, rare-class performance, and training stability. Enhanced with dual-rate exponential moving average smoothing and mixed-precision training, it ensures robustness and efficiency. Extensive experiments across Pascal VOC, COCO, and VG-256 demonstrate MAT-Agent's superiority: it achieves an mAP of 97.4 (vs. 96.2 for PAT-T), OF1 of 92.3, and CF1 of 91.4 on Pascal VOC; an mAP of 92.8 (vs. 92.0 for HSQ-CvN), OF1 of 88.2, and CF1 of 87.1 on COCO; and an mAP of 60.9, OF1 of 70.8, and CF1 of 61.1 on VG-256. With accelerated convergence and robust cross-domain generalization, MAT-Agent offers a scalable, intelligent solution for optimizing complex visual models, paving the way for adaptive deep learning advancements.