mlsanityData to 7 Oct 2026

MiniGPT-Reverse-Designing: Predicting Image Adjustments Utilizing MiniGPT-4

arXiv 2406.00971 ↗First posted 3 Jun 2024cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-Language Models (VLMs) have recently seen significant advancements through integrating with Large Language Models (LLMs). The VLMs, which process image and text modalities simultaneously, have demonstrated the ability to learn and understand the interaction between images and texts across various multi-modal tasks. Reverse designing, which could be defined as a complex vision-language task, aims to predict the edits and their parameters, given a source image, an edited version, and an optional high-level textual edit description. This task requires VLMs to comprehend the interplay between the source image, the edited version, and the optional textual context simultaneously, going beyond traditional vision-language tasks. In this paper, we extend and fine-tune MiniGPT-4 for the reverse designing task. Our experiments demonstrate the extensibility of off-the-shelf VLMs, specifically MiniGPT-4, for more complex tasks such as reverse designing. Code is available at this hrefhttps://github.com/VahidAz/MiniGPT-Reverse-Designingrepository.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 24 references that are in the map. First posted on arXiv 3 Jun 2024.

Builds on

18 of 24 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Enhancing Vision-Language Pre-training with Rich Supervisions5 Mar
  2. 2022A Survey of Vision-Language Pre-Trained Models18 Feb

Other references · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  7. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  8. 2023Large Language Modelscited by 774
  9. 2022Multimodal Learning with Transformers: A Surveycited by 399
  10. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 364
  11. 2017Universal Style Transfer via Feature Transformscited by 352
  12. 2023Multimodal Large Language Models: A Surveycited by 193
4 more
  1. 2017Image to Image Translation for Domain Adaptationcited by 171
  2. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  3. 2023XNLI: Explaining and Diagnosing NLI-based Visual Data Analysiscited by 18
  4. 2023Vision + Language Applications: A Surveycited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024An Introduction to Vision-Language Modeling
  2. 2026ReViCo: Unveiling the Limitations of VLMs in Visual Text Understanding via Error Correction
  3. 2025On the Limitations of Vision-Language Models in Understanding Image Transforms
  4. 2025Multilingual VLM Training: Adapting an English-Trained VLM to French
  5. 2026M-MiniGPT4: Multilingual VLLM Alignment via Translated Data
  6. 2025NanoVLMs: How small can we go and still make coherent Vision Language Models?
  7. 2025Adapting Vision-Language Models Without Labels: A Comprehensive Survey
  8. 2025Generalizing vision-language models to novel domains: A comprehensive survey