# RoboTron-Mani: All-in-One Multimodal Large Model for Robotic Manipulation

arXiv: https://arxiv.org/abs/2412.07215 · HTML: https://mlsanity.com/paper/2412.07215/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2024.

## Built on by (most cited first, 14 of 14)

- [Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications](https://mlsanity.com/paper/2510.07077/) (2025)
- [World Action Models: The Next Frontier in Embodied AI](https://mlsanity.com/paper/2605.12090/) (2026)
- [Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models](https://mlsanity.com/paper/2606.17846/) (2026)
- [Pure Vision Language Action (VLA) Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.19012/) (2025)
- [Agentic Robot: A Brain-Inspired Framework for Vision-Language-Action Models in Embodied Agents](https://mlsanity.com/paper/2505.23450/) (2025)
- [Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Model](https://mlsanity.com/paper/2605.14950/) (2026)
- [World Action Models: A Survey](https://mlsanity.com/paper/2606.20781/) (2026)
- [Survey of Vision-Language-Action Models for Embodied Manipulation](https://mlsanity.com/paper/2508.15201/) (2025)
- [ANNIE: Be Careful of Your Robots](https://mlsanity.com/paper/2509.03383/) (2025)
- [VLA-LPAF: Lightweight Perspective-Adaptive Fusion for Vision-Language-Action to Enable More Unconstrained Robotic Manipulation](https://mlsanity.com/paper/2509.18183/) (2025)
- [DLAM: Distributional Latent Actions with Temporal Constraints](https://mlsanity.com/paper/2607.27138/) (2026)
- [PASR: Pose-Aware 3D Shape Retrieval from Occluded Single Views](https://mlsanity.com/paper/2604.22658/) (2026)
- [SkiP: When to Skip and When to Refine for Efficient Robot Manipulation](https://mlsanity.com/paper/2605.15536/) (2026)
- [FabriVLA: A Lightweight Vision-Language-Action Model with Conformal Action Chunk Uncertainty](https://mlsanity.com/paper/2607.08575/) (2026)

## Abstract (arXiv)

Recently, robotics has advanced significantly through the integration of larger models and large-scale datasets. However, challenges remain in applying these models to 3D spatial interactions and managing data collection costs. To address these issues, we propose the multimodal robotic manipulation model RoboTron-Mani and the comprehensive dataset RoboData. RoboTron-Mani, on one hand, enhances 3D perception through camera parameters and occupancy supervision. On the other hand, it further incorporates Modality-Isolation-Mask and multimodal decoder blocks based on OpenFlamingo, improving modality fusion and fine-grained perception. RoboData integrats several publicly-available datasets, achieving the first fusion of multi-view images, camera parameters, depth maps, actions, and space alignment, which facilitates comprehensive learning from diverse robotic datasets and offers one complete evaluation system. Trained on RoboData, RoboTron-Mani is the first generalist policy that surpasses expert models, enabling simultaneous evaluation of all tasks across multiple datasets, rather than being limited to specific data or task selections. Specifically, RoboTron-Mani boosts manipulation performance by increasing the average sequence length on CALVIN from 1.7 to 3.5, enabling cross-embodiment generalization, and achieving state-of-the-art results on both simulated and real-world datasets.