# DAPE: Dynamic Non-uniform Alignment and Progressive Detail Enhancement Techniques for Improving the Performance of Efficient Visual Language Models

arXiv: https://arxiv.org/abs/2605.08902 · HTML: https://mlsanity.com/paper/2605.08902/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 9 May 2026.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices](https://mlsanity.com/paper/2004.02984/) (2020)
- [Exploring Visual Prompts for Adapting Large-Scale Models](https://mlsanity.com/paper/2203.17274/) (2022)
- [Real-time Scene Text Detection with Differentiable Binarization](https://mlsanity.com/paper/1911.08947/) (2019)
- [Shape Robust Text Detection with Progressive Scale Expansion Network](https://mlsanity.com/paper/1903.12473/) (2019)
- [SEED: Self-supervised Distillation For Visual Representation](https://mlsanity.com/paper/2101.04731/) (2021)
- [BERT-of-Theseus: Compressing BERT by Progressive Module Replacing](https://mlsanity.com/paper/2002.02925/) (2020)
- [Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusion](https://mlsanity.com/paper/2202.10304/) (2022)
- [Understanding and Improving Visual Prompting: A Label-Mapping Perspective](https://mlsanity.com/paper/2211.11635/) (2022)
- [Wasserstein Contrastive Representation Distillation](https://mlsanity.com/paper/2012.08674/) (2020)
- [Contrastive Distillation on Intermediate Representations for Language Model Compression](https://mlsanity.com/paper/2009.14167/) (2020)
- [Improving Visual Prompt Tuning for Self-supervised Vision Transformers](https://mlsanity.com/paper/2306.05067/) (2023)
- [Revisiting the Power of Prompt for Visual Tuning](https://mlsanity.com/paper/2402.02382/) (2024)
- [DOPRA: Decoding Over-accumulation Penalization and Re-allocation in Specific Weighting Layer](https://mlsanity.com/paper/2407.15130/) (2024)
- [MCA-LLaVA: Manhattan Causal Attention for Reducing Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2507.09184/) (2025)
- [Fourier Prompt Tuning for Modality-Incomplete Scene Segmentation](https://mlsanity.com/paper/2401.16923/) (2024)
- [Hallucination Begins Where Saliency Drops](https://mlsanity.com/paper/2601.20279/) (2026)
- [Context Tokens are Anchors: Understanding the Repetition Curse in dMLLMs from an Information Flow Perspective](https://mlsanity.com/paper/2601.20520/) (2026)
- [TextMamba: Scene Text Detector with Mamba](https://mlsanity.com/paper/2512.06657/) (2025)
- [C^2ROPE: Causal Continuous Rotary Positional Encoding for 3D Large Multimodal-Models Reasoning](https://mlsanity.com/paper/2602.10551/) (2026)
- [SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMs](https://mlsanity.com/paper/2602.22716/) (2026)
- [Reasoning Fails Where Step Flow Breaks](https://mlsanity.com/paper/2604.06695/) (2026)
- [ART: Attention Replacement Technique to Improve Factuality in LLMs](https://mlsanity.com/paper/2604.06393/) (2026)

## Abstract (arXiv)

In recent years, pre-trained visual-linguistic models have demonstrated tremendous potential, becoming a crucial foundational framework for numerous downstream tasks. However, the information density between text and images is not uniformly distributed. Existing methods often overlook the inherent and dynamic differences in information density and semantic scope between text tags and image blocks. These common uniform alignment strategies result in coarse-grained cross-modal interactions and loss of fine semantic details. Moreover, pursuing finer alignment typically requires substantial computational overhead, limiting practical model deployment. To address this challenge, this paper proposes a novel framework for dynamic cross-modal alignment with continuous detail introduction. First, we design a dynamically adaptive cross-modal matching mechanism that uses a learnable matching function to dynamically assign varying numbers and sizes of image tags to text tags of the same size but different information density, enabling more precise attention interaction. Second, we develop a continuous detail introduction module to progressively incorporate high-resolution visual feature enhancement into the alignment process. Extensive experiments across multiple benchmarks demonstrate significant improvements in the accuracy of various downstream tasks while reducing computational overhead.