# On Path to Multimodal Generalist: General-Level and General-Bench

arXiv: https://arxiv.org/abs/2505.04620 · HTML: https://mlsanity.com/paper/2505.04620/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- First posted on arXiv 7 May 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Built on by (most cited first, 19 of 19)

- [DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World](https://mlsanity.com/paper/2506.24102/) (2025)
- [UniVA: Universal Video Agent towards Open-Source Next-Generation Video Generalist](https://mlsanity.com/paper/2511.08521/) (2025)
- [Circle-RoPE: Cone-like Decoupled Rotary Positional Embedding for Large Vision-Language Models](https://mlsanity.com/paper/2505.16416/) (2025)
- [Mixed-R1: Unified Reward Perspective For Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2505.24164/) (2025)
- [Dr.V: A Hierarchical Perception-Temporal-Cognition Framework to Diagnose Video Hallucination by Fine-grained Spatial-Temporal Grounding](https://mlsanity.com/paper/2509.11866/) (2025)
- [UniM: A Unified Any-to-Any Interleaved Multimodal Benchmark](https://mlsanity.com/paper/2603.05075/) (2026)
- [Dense360: Dense Understanding from Omnidirectional Panoramas](https://mlsanity.com/paper/2506.14471/) (2025)
- [MuSLR: Multimodal Symbolic Logical Reasoning](https://mlsanity.com/paper/2509.25851/) (2025)
- [Benchmark Everything Everywhere All at Once](https://mlsanity.com/paper/2606.06462/) (2026)
- [Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearning](https://mlsanity.com/paper/2605.05909/) (2026)
- [Adapter-Augmented Bandits for Online Multi-Constrained Multi-Modal Inference Scheduling](https://mlsanity.com/paper/2603.06403/) (2026)
- [Multimodal Evaluation of Russian-language Architectures](https://mlsanity.com/paper/2511.15552/) (2025)
- [AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMs](https://mlsanity.com/paper/2606.07643/) (2026)
- [In-Context Learning for Robots: Methods and Applications](https://mlsanity.com/paper/2609.36012/) (2026)
- [Ostrakon-VL: Towards Domain-Expert MLLM for Food-Service and Retail Stores](https://mlsanity.com/paper/2601.21342/) (2026)
- [Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinking](https://mlsanity.com/paper/2602.21435/) (2026)
- [Orthogonal Spatial-temporal Distributional Transfer for 4D Generation](https://mlsanity.com/paper/2603.05081/) (2026)
- [OmniAssistBench: Assistant-style Interaction Benchmark for Omni-LLMs](https://mlsanity.com/paper/2608.21360/) (2026)
- [Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model](https://mlsanity.com/paper/2609.18323/) (2026)

## Abstract (arXiv)

The Multimodal Large Language Model (MLLM) is currently experiencing rapid growth, driven by the advanced capabilities of LLMs. Unlike earlier specialists, existing MLLMs are evolving towards a Multimodal Generalist paradigm. Initially limited to understanding multiple modalities, these models have advanced to not only comprehend but also generate across modalities. Their capabilities have expanded from coarse-grained to fine-grained multimodal understanding and from supporting limited modalities to arbitrary ones. While many benchmarks exist to assess MLLMs, a critical question arises: Can we simply assume that higher performance across tasks indicates a stronger MLLM capability, bringing us closer to human-level AI? We argue that the answer is not as straightforward as it seems. This project introduces General-Level, an evaluation framework that defines 5-scale levels of MLLM performance and generality, offering a methodology to compare MLLMs and gauge the progress of existing systems towards more robust multimodal generalists and, ultimately, towards AGI. At the core of the framework is the concept of Synergy, which measures whether models maintain consistent capabilities across comprehension and generation, and across multiple modalities. To support this evaluation, we present General-Bench, which encompasses a broader spectrum of skills, modalities, formats, and capabilities, including over 700 tasks and 325,800 instances. The evaluation results that involve over 100 existing state-of-the-art MLLMs uncover the capability rankings of generalists, highlighting the challenges in reaching genuine AI. We expect this project to pave the way for future research on next-generation multimodal foundation models, providing a robust infrastructure to accelerate the realization of AGI. Project page: https://generalist.top/