mlsanityData to 14 Oct 2026

Aligning by Misaligning: Boundary-aware Curriculum Learning for Multimodal Alignment

arXiv 2511.08399 ↗First posted 11 Nov 2025cs.LG cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Most multimodal models treat every negative pair alike, ignoring the ambiguous negatives that differ from the positive by only a small detail. We propose Boundary-Aware Curriculum with Local Attention (BACL), a lightweight add-on that turns these borderline cases into a curriculum signal. A Boundary-aware Negative Sampler gradually raises difficulty, while a Contrastive Local Attention loss highlights where the mismatch occurs. The two modules are fully differentiable and work with any off-the-shelf dual encoder. Theory predicts a fast O(1/n) error rate; practice shows up to +32% R@1 over CLIP and new SOTA on four large-scale benchmarks, all without extra labels.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 41 of its 74 references that are in the map. First posted on arXiv 11 Nov 2025.

Builds on

41 of 74 references.

References · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  5. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  6. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  7. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  8. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  9. 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 507
  10. 2019UNITER: UNiversal Image-TExt Representation Learningcited by 228
  11. 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 169
  12. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 166
29 more
  1. 2024FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Modelscited by 113
  2. 2023NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Spacecited by 90
  3. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 89
  4. 2024A Touch, Vision, and Language Dataset for Multimodal Alignmentcited by 69
  5. 2014Curriculum Learning of Multiple Taskscited by 66
  6. 2024Gramian Multimodal Representation Learning and Alignmentcited by 54
  7. 2024What to align in multimodal contrastive learning?cited by 47
  8. 2023MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learningcited by 41
  9. 2022Audio Retrieval with WavText5K and CLAP Trainingcited by 35
  10. 2025Understanding the Emergence of Multimodal Representation Alignmentcited by 31
  11. 2024Prior Knowledge Integration via LLM Encoding and Pseudo Event Regulation for Video Moment Retrievalcited by 30
  12. 2024DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Drivingcited by 29
  13. 2025Removal of Hallucination on Hallucination: Debate-Augmented RAGcited by 29
  14. 2024Swift Sampler: Efficient Learning of Sampler by 10 Parameterscited by 25
  15. 2025Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMscited by 25
  16. 2025Reasoning-Enhanced Domain-Adaptive Pretraining of Multimodal Large Language Models for Short Video Content Governancecited by 16
  17. 2025Audio-Enhanced Vision-Language Modeling with Latent Space Broadening for High Quality Data Expansioncited by 15
  18. 2025Convex Relaxation for Robust Vanishing Point Estimation in Manhattan Worldcited by 15
  19. 2019How humans learn and represent networkscited by 10
  20. 2024TriSampler: A Better Negative Sampling Principle for Dense Retrievalcited by 10
  21. 2023Dual-view Curricular Optimal Transport for Cross-lingual Cross-modal Retrievalcited by 9
  22. 2025PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learningcited by 9
  23. 2025C3-OWD: A Curriculum Cross-modal Contrastive Learning Framework for Open-World Detectioncited by 5
  24. 2025Renaissance of RNNs in Streaming Clinical Time Series: Compact Recurrence Remains Competitive with Transformerscited by 5
  25. 2025Curiosity Meets Cooperation: A Game-Theoretic Approach to Long-Tail Multi-Label Learningcited by 4
  26. 2025CLIP-SENet: CLIP-based Semantic Enhancement Network for Vehicle Re-identificationcited by 3
  27. 2025DOA: A Degeneracy Optimization Agent with Adaptive Pose Compensation Capability based on Deep Reinforcement Learningcited by 3
  28. 2025Lightweight Baselines for Medical Abstract Classification: DistilBERT with Cross-Entropy as a Strong Defaultcited by 3
  29. 2025RegionMed-CLIP: A Region-Aware Multimodal Contrastive Learning Pre-trained Model for Medical Image Understandingcited by 1

Built on by

Not cited yet.