# FusionMMT: A Unified Multimodal and Multitask Learning Framework for Nuclear Fusion

arXiv: https://arxiv.org/abs/2609.26095 · HTML: https://mlsanity.com/paper/2609.26095/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 46 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Builds on (most cited first, 12 of 12)

- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Multi-Time Attention Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/2101.10318/) (2021)
- [Hybrid deep learning architecture for general disruption prediction across tokamaks](https://mlsanity.com/paper/2007.01401/) (2020)
- [Selective Task Group Updates for Multi-Task Optimization](https://mlsanity.com/paper/2502.11986/) (2025)
- [M3Net: Multimodal Multi-task Learning for 3D Detection, Segmentation, and Occupancy Prediction in Autonomous Driving](https://mlsanity.com/paper/2503.18100/) (2025)
- [MMTL-UniAD: A Unified Framework for Multimodal and Multi-Task Learning in Assistive Driving Perception](https://mlsanity.com/paper/2504.02264/) (2025)
- [UniAV: Unified Audio-Visual Perception for Multi-Task Video Event Localization](https://mlsanity.com/paper/2404.03179/) (2024)
- [Resolving Token-Space Gradient Conflicts: Token Space Manipulation for Transformer-Based Multi-Task Learning](https://mlsanity.com/paper/2507.07485/) (2025)
- [TokaMind: A Multi-Modal Transformer Foundation Model for Tokamak Plasma Dynamics](https://mlsanity.com/paper/2602.15084/) (2026)
- [TokaMark: A Comprehensive Benchmark for MAST Tokamak Plasma Models](https://mlsanity.com/paper/2602.10132/) (2026)
- [Uni-Hema: Unified Model for Digital Hematopathology](https://mlsanity.com/paper/2511.13889/) (2025)

## Abstract (arXiv)

With the growing global demand for energy, nuclear fusion has emerged as a promising direction for future clean energy. Tokamaks represent one of the leading approaches to magnetic-confinement fusion. Achieving high-performance, long-pulse, and steady-state operation requires effective diagnosis of plasma states. However, existing intelligent diagnostic methods are largely limited to either multimodal single-task or unimodal multitask learning, while a unified multimodal multitask learning framework remains underexplored. To address this gap, we construct EAST-VTD640, a multimodal multitask dataset that integrates vision and time-series diagnostics from 640 EAST shots for disruption prediction, edge-localized mode (ELM) recognition, and H98 regression. On this basis, we present FusionMMT, the first unified multimodal multitask framework for intelligent tokamak plasma diagnostics. FusionMMT employs multi-scale, time-aware, and variable-aware modeling to handle heterogeneous sampling rates and the high computational cost of high-frequency sequences. It further combines task-adaptive multimodal fusion with progressive multitask optimization to learn shared and task-specific representations while mitigating cross-task conflicts and optimization imbalance. Extensive experiments on EAST-VTD640 show that FusionMMT outperforms representative multimodal multitask methods across disruption prediction, ELM recognition, and H98 regression. The source code will be released on https://github.com/Event-AHU/OpenFusion