AviationLMM: A Large Multimodal Foundation Model for Civil Aviation
Not on the map. Why?
Abstract (arXiv)
Civil aviation is a cornerstone of global transportation and commerce, and ensuring its safety, efficiency and customer satisfaction is paramount. Yet conventional Artificial Intelligence (AI) solutions in aviation remain siloed and narrow, focusing on isolated tasks or single modalities. They struggle to integrate heterogeneous data such as voice communications, radar tracks, sensor streams and textual reports, which limits situational awareness, adaptability, and real-time decision support. This paper introduces the vision of AviationLMM, a Large Multimodal foundation Model for civil aviation, designed to unify the heterogeneous data streams of civil aviation and enable understanding, reasoning, generation and agentic applications. We firstly identify the gaps between existing AI solutions and requirements. Secondly, we describe the model architecture that ingests multimodal inputs such as air-ground voice, surveillance, on-board telemetry, video and structured texts, and performs cross-modal alignment and fusion, and produces flexible outputs ranging from situation summaries and risk alerts to predictive diagnostics and multimodal incident reconstructions. In order to fully realize this vision, we identify key research opportunities to address, including data acquisition, alignment and fusion, pretraining, reasoning, trustworthiness, privacy, robustness to missing modalities, and synthetic scenario generation. By articulating the design and challenges of AviationLMM, we aim to boost the civil aviation foundation model progress and catalyze coordinated research efforts toward an integrated, trustworthy and privacy-preserving aviation AI ecosystem.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 51 references that are in the map. First posted on arXiv 14 Jan 2026.
Builds on
36 of 51 references.
References · 36
- 2023Visual Instruction Tuningcited by 7,792
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
- 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
24 more
- 2022Towards Reasoning in Large Language Models: A Surveycited by 593
- 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 391
- 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
- 2020Spatial-Temporal Transformer Networks for Traffic Flow Forecastingcited by 179
- 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
- 2025TerraMind: Large-Scale Generative Multimodality for Earth Observationcited by 101
- 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 89
- 2023FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learningcited by 59
- 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Modelscited by 51
- 2024Conformal Alignment: Knowing When to Trust Foundation Models with Guaranteescited by 50
- 2023Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Modelscited by 49
- 2023VIP5: Towards Multimodal Foundation Models for Recommendationcited by 45
- 2024RF-Diffusion: Radio Signal Generation via Time-Frequency Diffusioncited by 41
- 2023Parameter-efficient Tuning of Large-scale Multimodal Foundation Modelcited by 23
- 2024HEMM: Holistic Evaluation of Multimodal Foundation Modelscited by 21
- 2024Deep Correlated Prompting for Visual Recognition with Missing Modalitiescited by 17
- 2025MMDT: Decoding the Trustworthiness and Safety of Multimodal Foundation Modelscited by 14
- 2024AccidentGPT: Large Multi-Modal Foundation Model for Traffic Accident Analysiscited by 10
- 2024SeaMo: A Season-Aware Multimodal Foundation Model for Remote Sensingcited by 10
- 2023AviationGPT: A Large Language Model for the Aviation Domaincited by 9
- 2024Mixed-Query Transformer: A Unified Image Segmentation Architecturecited by 2
- 2025AviationLLM: An LLM-based Knowledge System for Aviation Trainingcited by 2
- 2022Incomplete Multi-View Weak-Label Learning with Noisy Features and Imbalanced Labelscited by 1
- 2025Multimodal Foundation Model for Cross-Modal Retrieval and Activity Recognition Taskscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025CAMB: A comprehensive industrial LLM benchmark on civil aviation maintenance
- 2026PilotBench: A Benchmark for General Aviation Agents with Safety Constraints
- 2025Multimodal Large Language Models-Enabled UAV Swarm: Towards Efficient and Intelligent Autonomous Aerial Systems
- 2026AeroCopilotBench: A Two-Tier Benchmark for Evaluating LLM Agents as Aviation Copilots in an Interactive Virtual Cockpit Environment
- 2026Can Large Language Models Explain Flight Safety Events? A Prior-Guided Semantic LLM-based Approach
- 2026Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System
- 2025Enhancing Low-Altitude Airspace Security: MLLM-Enabled UAV Intent Recognition
- 2025UAVBench: An Open Benchmark Dataset for Autonomous and Agentic AI UAV Systems via LLM-Generated Flight Scenarios