mlsanityData to 14 Oct 2026

Dynamic Context Adapters: Efficiently Infusing History into Vision-and-Language Models

arXiv 2608.10525 ↗First posted 11 Aug 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

Historical context integration presents a fundamental challenge for Vision-Language Models (VLMs) in sequential decision-making tasks. Current VLMs process visual inputs independently, which creates critical limitations for downstream applications that require temporal understanding. Direct incorporation of historical frames into Transformer inputs produces quadratic attention complexity and excessive memory consumption. Existing approaches suffer from significant drawbacks: computational inflation or substantial information loss through temporal compression. To address these challenges, we introduce Dynamic Context Adapter (DCA), a novel context injection approach for pretrained VLMs. Our method employs fixed-size, dynamically compressed memory to preserve historical semantics without frame concatenation. DCA bridges static VLMs and recurrent policies and enables memory capabilities in pretrained models while maintaining computational efficiency. DCA achieves over 25% reduction in attention FLOPs and 13% memory savings while improving performance on long-horizon tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 65 of its 69 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

65 of 69 references.

References · 65

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2024The Llama 3 Herd of Modelscited by 14,715
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  9. 2023Visual Instruction Tuningcited by 7,869
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  12. 2023Qwen Technical Reportcited by 3,344
38 more
  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  3. 2015VQA: Visual Question Answeringcited by 2,506
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  5. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  6. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  7. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  8. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  9. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 883
  10. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  11. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
  12. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  13. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 439
  14. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  15. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 339
  16. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  17. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  18. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  19. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  20. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
  21. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  22. 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 214
  23. 2018Semi-parametric Topological Memory for Navigationcited by 209
  24. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  25. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  26. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  27. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 138
  28. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 132
  29. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 130
  30. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 129
  31. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  32. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
  33. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
  34. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 98
  35. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 94
  36. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 86
  37. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  38. 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 86

50 most cited of 65 All 65 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TempoFit: Plug-and-Play Layer-Wise Temporal KV Memory for Long-Horizon Vision-Language-Action Manipulation
  2. 2026NativeMEM: Native Memory Compression for Long-Horizon Robotic Manipulation
  3. 2024A-VL: Adaptive Attention for Large Vision-Language Models
  4. 2026V-DyKnow: A Dynamic Benchmark for Time-Sensitive Knowledge in Vision Language Models
  5. 2026FlashBack: Knowing When to Remember in Streaming Vision-Language Models
  6. 2026UniMem: Unifying Multimodal Memory and Control for Vision-Language-Action Models
  7. 2026Explicit Language Memory for Long-Horizon Planning in Vision-Language-Action Models
  8. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects