mlsanityData to 7 Oct 2026

JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation

arXiv 2605.04128 ↗First posted 5 May 2026cs.GR cs.AI cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

We present JoyAI-Image, a unified multimodal foundation model for visual understanding, text-to-image generation, and instruction-guided image editing. JoyAI-Image couples a spatially enhanced Multimodal Large Language Model (MLLM) with a Multimodal Diffusion Transformer (MMDiT), allowing perception and generation to interact through a shared multimodal interface. Around this architecture, we build a scalable training recipe that combines unified instruction tuning, long-text rendering supervision, spatially grounded data, and both general and spatial editing signals. This design gives the model broad multimodal capability while strengthening geometry-aware reasoning and controllable visual synthesis. Experiments across understanding, generation, long-text rendering, and editing benchmarks show that JoyAI-Image achieves state-of-the-art or highly competitive performance. More importantly, the bidirectional loop between enhanced understanding, controllable spatial editing, and novel-view-assisted reasoning enables the model to move beyond general visual competence toward stronger spatial intelligence. These results suggest a promising path for unified visual models in downstream applications such as vision-language-action systems and world models.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 79 of its 95 references that are in the map. First posted on arXiv 5 May 2026.

Builds on

79 of 95 references.

References · 79

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2022Flow Matching for Generative Modelingcited by 5,426
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  8. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  10. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  11. 2025Qwen3-VL Technical Reportcited by 2,534
  12. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
38 more
  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  4. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  5. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  6. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  7. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  8. 2025Qwen-Image Technical Reportcited by 973
  9. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  10. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  11. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  12. 2025SAM 3: Segment Anything with Conceptscited by 835
  13. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  14. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  15. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  16. 2024Genie: Generative Interactive Environmentscited by 706
  17. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  18. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
  19. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  20. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  21. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  22. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  23. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
  24. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  25. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  26. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  27. 2024OmniGen: Unified Image Generationcited by 388
  28. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  29. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  30. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 326
  31. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  32. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
  33. 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 247
  34. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  35. 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
  36. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
  37. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
  38. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 10.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Exploring Spatial Intelligence from a Generative Perspective
  2. 2026Spatial Chain-of-Thought: Bridging Understanding and Generation Models for Spatial Reasoning Generation
  3. 2026SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning
  4. 2025MIGE: Mutually Enhanced Multimodal Instruction-Based Image Generation and Editing
  5. 2026Think, Plan, Paint: Layout-Aware Reasoning for Controllable Image Generation in Unified Models
  6. 2025Query-Kontext: An Unified Multimodal Model for Image Generation and Editing
  7. 2026Forge-and-Quench: Enhancing Image Generation for Higher Fidelity in Unified Multimodal Models
  8. 2026Thinking with Novel Views: A Systematic Analysis of Generative-Augmented Spatial Intelligence