mlsanityData to 14 Oct 2026

Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision

arXiv 2504.04903 ↗First posted 7 Apr 2025cs.CV cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

We present Lunima-OmniLV (abbreviated as OmniLV), a universal multimodal multi-task framework for low-level vision that addresses over 100 sub-tasks across four major categories: image restoration, image enhancement, weak-semantic dense prediction, and stylization. OmniLV leverages both textual and visual prompts to offer flexible and user-friendly interactions. Built on Diffusion Transformer (DiT)-based generative priors, our framework supports arbitrary resolutions -- achieving optimal performance at 1K resolution -- while preserving fine-grained details and high fidelity. Through extensive experiments, we demonstrate that separately encoding text and visual instructions, combined with co-training using shallow feature control, is essential to mitigate task ambiguity and enhance multi-task generalization. Our findings also reveal that integrating high-level generative tasks into low-level vision models can compromise detail-sensitive restoration. These insights pave the way for more robust and generalizable low-level vision systems.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 86 of its 121 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

86 of 121 references.

References · 86

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023Segment Anythingcited by 8,234
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  11. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  12. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
38 more
  1. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  3. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  4. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  5. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  6. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  7. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,358
  8. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  9. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,224
  10. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  11. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  12. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  13. 2021Multi-Stage Progressive Image Restorationcited by 816
  14. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  15. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 727
  16. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  17. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolutioncited by 489
  18. 2017Benchmarking Single Image Dehazing and Beyondcited by 447
  19. 2022Activating More Pixels in Image Super-Resolution Transformercited by 438
  20. 2024OmniGen: Unified Image Generationcited by 394
  21. 2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 349
  22. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
  23. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 324
  24. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 318
  25. 2020Learning Enriched Features for Real Image Restoration and Enhancementcited by 312
  26. 2022MAXIM: Multi-Axis MLP for Image Processingcited by 302
  27. 2024Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wildcited by 286
  28. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
  29. 2020Multi-Scale Progressive Fusion Network for Single Image Derainingcited by 260
  30. 2023Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylizationcited by 254
  31. 2024One-Step Effective Diffusion Network for Real-World Image Super-Resolutioncited by 246
  32. 2022Towards Robust Blind Face Restoration with Codebook Lookup Transformercited by 231
  33. 2023Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancementcited by 231
  34. 2017Attentive Generative Adversarial Network for Raindrop Removal from a Single Imagecited by 204
  35. 2017DesnowNet: Context-Aware Deep Network for Snow Removalcited by 200
  36. 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 193
  37. 2019Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learningcited by 157
  38. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137

50 most cited of 86 All 86 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 4.

14 papers.

Sort:

By area:

  1. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  2. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 88
  3. 2025Does FLUX Already Know How to Perform Physically Plausible Image Composition?cited by 32
  4. 2025Factuality Matters: When Image Generation and Editing Meet Structured Visualscited by 20
  5. 2025PICABench: How Far Are We from Physically Realistic Image Editing?cited by 16
  6. 2025FoundIR-v2: Optimizing Pre-Training Data Mixtures for Image Restoration Foundation Modelcited by 9
  7. 2026From Scale to Speed: Adaptive Test-Time Scaling for Image Editingcited by 8
  8. 2026TIR-Agent: Training an Explorative and Efficient Agent for Image Restorationcited by 5
  9. 2025FAPE-IR: Frequency-Aware Planning and Execution Framework for All-in-One Image Restorationcited by 4
  10. 2025Omni-Weather: A Unified Multimodal Model for Weather Radar Understanding and Generationcited by 2
  11. 2025FramePrompt: In-context Controllable Animation with Zero Structural Changescited by 1
  12. 2025SynWeather: Weather Observation Data Synthesis across Multiple Regions and Variables via a General Diffusion Transformercited by 0
2 more
  1. 2026Rethinking Where to Edit: Task-Aware Localization for Instruction-Based Image Editingcited by 0
  2. 2026Universal Image Restoration via Internalized Chain-of-Thought Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025OpenVision 2: A Family of Generative Pretrained Visual Encoders for Multimodal Learning
  2. 2026UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling
  3. 2025From Enhancement to Understanding: Build a Generalized Bridge for Low-light Vision via Semantically Consistent Unsupervised Fine-tuning
  4. 2025UniFusion: Vision-Language Model as Unified Encoder in Image Generation
  5. 2025UniModel: A Visual-Only Framework for Unified Multimodal Understanding and Generation
  6. 2026Looping Back to Move Forward: Recursive Transformers for Efficient and Flexible Large Multimodal Models
  7. 2024LM4LV: A Frozen Large Language Model for Low-level Vision Tasks
  8. 2023InstructCV: Instruction-Tuned Text-to-Image Diffusion Models as Vision Generalists