mlsanityData to 14 Oct 2026

NeoBabel: A Multilingual Open Tower for Visual Generation

arXiv 2507.06137 ↗First posted 8 Jul 2025cs.CL cs.AI cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Text-to-image generation advancements have been predominantly English-centric, creating barriers for non-English speakers and perpetuating digital inequities. While existing systems rely on translation pipelines, these introduce semantic drift, computational overhead, and cultural misalignment. We introduce NeoBabel, a novel multilingual image generation framework that sets a new Pareto frontier in performance, efficiency and inclusivity, supporting six languages: English, Chinese, Dutch, French, Hindi, and Persian. The model is trained using a combination of large-scale multilingual pretraining and high-resolution instruction tuning. To evaluate its capabilities, we expand two English-only benchmarks to multilingual equivalents: m-GenEval and m-DPG. NeoBabel achieves state-of-the-art multilingual performance while retaining strong English capability, scoring 0.75 on m-GenEval and 0.68 on m-DPG. Notably, it performs on par with leading models on English tasks while outperforming them by +0.11 and +0.09 on multilingual benchmarks, even though these models are built on multilingual base LLMs. This demonstrates the effectiveness of our targeted alignment training for preserving and extending crosslingual generalization. We further introduce two new metrics to rigorously assess multilingual alignment and robustness to code-mixed prompts. Notably, NeoBabel matches or exceeds English-only models while being 2-4x smaller. We release an open toolkit, including all code, model checkpoints, a curated dataset of 124M multilingual text-image pairs, and standardized multilingual evaluation protocols, to advance inclusive AI research. Our work demonstrates that multilingual capability is not a trade-off but a catalyst for improved robustness, efficiency, and cultural fidelity in generative AI.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 85 of its 88 references that are in the map. First posted on arXiv 8 Jul 2025.

Builds on

85 of 88 references.

References · 85

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Segment Anythingcited by 8,234
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  9. 2022Classifier-Free Diffusion Guidancecited by 5,847
  10. 2022Flow Matching for Generative Modelingcited by 5,702
  11. 2025Qwen2.5-VL Technical Reportcited by 5,587
  12. 2024Qwen2.5 Technical Reportcited by 4,942
38 more
  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  8. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  9. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  10. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  11. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  12. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  13. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  14. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  15. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  16. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  17. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  18. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  19. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  20. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  21. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 615
  22. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 502
  23. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  24. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  25. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  26. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  27. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  28. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  29. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 302
  30. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
  31. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  32. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  33. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 282
  34. 2020Query-Key Normalization for Transformerscited by 280
  35. 2025Transfer between Modalities with MetaQueriescited by 236
  36. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  37. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 205
  38. 2023Any-to-Any Generation via Composable Diffusioncited by 204

50 most cited of 85 All 85 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.