JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation
Not on the map. Why?
Abstract (arXiv)
We present JoyAI-Image, a unified multimodal foundation model for visual understanding, text-to-image generation, and instruction-guided image editing. JoyAI-Image couples a spatially enhanced Multimodal Large Language Model (MLLM) with a Multimodal Diffusion Transformer (MMDiT), allowing perception and generation to interact through a shared multimodal interface. Around this architecture, we build a scalable training recipe that combines unified instruction tuning, long-text rendering supervision, spatially grounded data, and both general and spatial editing signals. This design gives the model broad multimodal capability while strengthening geometry-aware reasoning and controllable visual synthesis. Experiments across understanding, generation, long-text rendering, and editing benchmarks show that JoyAI-Image achieves state-of-the-art or highly competitive performance. More importantly, the bidirectional loop between enhanced understanding, controllable spatial editing, and novel-view-assisted reasoning enables the model to move beyond general visual competence toward stronger spatial intelligence. These results suggest a promising path for unified visual models in downstream applications such as vision-language-action systems and world models.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 79 of its 95 references that are in the map. First posted on arXiv 5 May 2026.
Builds on
79 of 95 references.
References · 79
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
38 more
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
- 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
- 2025Qwen-Image Technical Reportcited by 973
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
- 2025SAM 3: Segment Anything with Conceptscited by 835
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
- 2024Genie: Generative Interactive Environmentscited by 706
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
- 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
- 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
- 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
- 2024OmniGen: Unified Image Generationcited by 388
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
- 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
- 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 326
- 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
- 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
- 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 247
- 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
- 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
- 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
- 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
- 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
50 most cited of 79 All 79 (JSON)
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budgetcited by 5
- 2026CPI-Bench: A Comprehensive, Practical and Intelligent Benchmark for Real-World Image Editingcited by 2
- 2026Exploring the Performance Frontier of Compact Unified Image Generation Modelscited by 1
- 2026Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillationcited by 0
- 2026Oxygen-TryOn: Fashion-Native Foundation Model for Any-item Virtual Try-Oncited by 0
- 2026Puffin-World: Scaling a Unified Multimodal Model with Native 3D World Statescited by 0
- 2026KwaiMind Technical Reportcited by 0
- 2026VibeEdit: Image Editing with Canvas Instructionscited by 0
- 2026From Prompting to Composing: A Spatial Canvas Interface for Poster Generationcited by 0
- 2026Reasoning-Informed Visual Editingcited by 0
- 2026Reasoning-Informed Visual Editing8 Oct 2026
- 2026From Prompting to Composing: A Spatial Canvas Interface for Poster Generation8 Oct 2026
- 2026VibeEdit: Image Editing with Canvas Instructions8 Oct 2026
- 2026KwaiMind Technical Report22 Sep 2026
- 2026Puffin-World: Scaling a Unified Multimodal Model with Native 3D World States3 Sep 2026
- 2026Exploring the Performance Frontier of Compact Unified Image Generation Models20 Aug 2026
- 2026CPI-Bench: A Comprehensive, Practical and Intelligent Benchmark for Real-World Image Editing14 Aug 2026
- 2026Oxygen-TryOn: Fashion-Native Foundation Model for Any-item Virtual Try-On23 Jul 2026
- 2026Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budget14 Jul 2026
- 2026Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillation2 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026Exploring Spatial Intelligence from a Generative Perspective
- 2026Spatial Chain-of-Thought: Bridging Understanding and Generation Models for Spatial Reasoning Generation
- 2026SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning
- 2025MIGE: Mutually Enhanced Multimodal Instruction-Based Image Generation and Editing
- 2026Think, Plan, Paint: Layout-Aware Reasoning for Controllable Image Generation in Unified Models
- 2025Query-Kontext: An Unified Multimodal Model for Image Generation and Editing
- 2026Forge-and-Quench: Enhancing Image Generation for Higher Fidelity in Unified Multimodal Models
- 2026Thinking with Novel Views: A Systematic Analysis of Generative-Augmented Spatial Intelligence