mlsanityData to 14 Oct 2026

Cohort-Based Active Modality Acquisition

arXiv 2505.16791 ↗First posted 22 May 2025cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Real-world multimodal machine learning often faces missing, costly-to-acquire modalities, raising the problem of which samples to prioritize for additional acquisition under a budget. Prior work mainly studies per-sample or training-time acquisition while test-time, cohort-level acquisition is less explored. We propose Cohort-based Active Modality Acquisition (CAMA), a novel test-time cohort-level modality acquisition setting, and introduce imputation-based acquisition strategies that estimate the expected utility of acquiring a missing modality, along with upper-bound heuristics for benchmarking. Experiments on datasets with up to 15 modalities demonstrate that our proposed imputation-based strategies can more effectively guide the acquisition of an additional modality for selected samples compared with methods relying solely on pre-acquisition information, entropy-based guidance, or random selection. We showcase the real-world relevance and scalability of our method by demonstrating its ability to guide the acquisition of proteomics data for disease prediction in a large prospective cohort, the UK Biobank (UKB). Our work provides an effective approach for optimizing modality acquisition at the cohort level, enabling more effective use of resources in constrained settings.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 41 of its 66 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

41 of 66 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  8. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  9. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 610
  10. 2020A Survey of Deep Active Learningcited by 554
  11. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
  12. 2024The Road Less Scheduledcited by 163
29 more
  1. 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 153
  2. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 92
  3. 2018EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAEcited by 87
  4. 2021Generalized Multimodal ELBOcited by 87
  5. 2022Integrated multimodal artificial intelligence framework for healthcare applicationscited by 71
  6. 2019Bayesian Generative Active Deep Learningcited by 57
  7. 2024A Closer Look at AUROC and AUPRC under Class Imbalancecited by 57
  8. 2020Active Feature Acquisition with Generative Surrogate Modelscited by 54
  9. 2024A Survey on Deep Active Learning: Recent Advances and New Frontierscited by 54
  10. 2022A Framework and Benchmark for Deep Batch Active Learning for Regressioncited by 49
  11. 2024Contrasting with Symile: Simple Model-Agnostic Representation Learning for Unlimited Modalitiescited by 24
  12. 2019Classification with Costly Features as a Sequential Decision-Making Problemcited by 23
  13. 2022Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlocited by 19
  14. 2023Acquisition Conditioned Oracle for Nongreedy Active Feature Acquisitioncited by 19
  15. 2023Algorithm Selection for Deep Active Learning with Imbalanced Datasetscited by 18
  16. 2025A Survey on Active Feature Acquisition Strategiescited by 17
  17. 2022Active Acquisition for Multimodal Temporal Data: A Challenging Decision-Making Taskcited by 13
  18. 2023Towards Balanced Active Learning for Multimodal Classificationcited by 9
  19. 2023Evaluation of Active Feature Acquisition Methods for Time-varying Feature Settingscited by 9
  20. 2019Multi-modal Active Learning From Human Data: A Deep Reinforcement Learning Approachcited by 8
  21. 2023Score-Based Multimodal Autoencodercited by 7
  22. 2023Modality-invariant Visual Odometry for Embodied Visioncited by 6
  23. 2025Reducing Aleatoric and Epistemic Uncertainty through Multi-modal Data Acquisitioncited by 5
  24. 2024Generative Active Learning for Image Synthesis Personalizationcited by 4
  25. 2020Peri-Diagnostic Decision Support Through Cost-Efficient Feature Acquisition at Test-Timecited by 3
  26. 2023Generator Assisted Mixture of Experts For Feature Acquisition in Batchcited by 2
  27. 2024MINT: A wrapper to make multi-modal and multi-image AI models interactivecited by 2
  28. 2025To Measure or Not: A Cost-Sensitive, Selective Measuring Environment for Agricultural Management Decisions with Reinforcement Learningcited by 2
  29. 2022MAViC: Multimodal Active Learning for Video Captioningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2025Fusion or Confusion? Multimodal Complexity Is Not All You Needcited by 5
  2. 2026Hidden in the Multiplicative Interaction: Uncovering Fragility in Multimodal Contrastive Learningcited by 2
  3. 2026Beyond Objective Expressivity: Geometry Preservation in Multimodal Contrastive Learningcited by 1
  4. 2026GraphSelect for Budgeted Representation Selection in Multimodal Graph Inferencecited by 0
  5. 2026Prior-Fitted and Frozen Scorers for Multimodal Contrastive Learningcited by 0