mlsanityData to 7 Oct 2026

AutoGluon-Multimodal (AutoMM): Supercharging Multimodal AutoML with Foundation Models

arXiv 2404.16233 ↗First posted 24 Apr 2024cs.LG cs.AIcited by 29

Not on the map. Why?

Abstract (arXiv)

AutoGluon-Multimodal (AutoMM) is introduced as an open-source AutoML library designed specifically for multimodal learning. Distinguished by its exceptional ease of use, AutoMM enables fine-tuning of foundation models with just three lines of code. Supporting various modalities including image, text, and tabular data, both independently and in combination, the library offers a comprehensive suite of functionalities spanning classification, regression, object detection, semantic matching, and image segmentation. Experiments across diverse datasets and tasks showcases AutoMM's superior performance in basic classification and regression tasks compared to existing AutoML tools, while also demonstrating competitive results in advanced tasks, aligning with specialized toolboxes designed for such purposes.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 71 of its 128 references that are in the map. First posted on arXiv 24 Apr 2024.

Builds on

71 of 128 references.

References · 71

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  10. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  11. 2023Segment Anythingcited by 8,158
  12. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
38 more
  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  5. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  6. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  9. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  10. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  11. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  12. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  13. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  14. 2017Mixed Precision Trainingcited by 1,135
  15. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  16. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  17. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  18. 2020Image Segmentation Using Deep Learning: A Surveycited by 659
  19. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  20. 2017DOTA: A Large-scale Dataset for Object Detection in Aerial Imagescited by 622
  21. 2019Kvasir-SEG: A Segmented Polyp Datasetcited by 588
  22. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  23. 2019Object Detection in 20 Years: A Surveycited by 528
  24. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
  25. 2017Structure-measure: A New Way to Evaluate Foreground Mapscited by 480
  26. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
  27. 2019PointRend: Image Segmentation as Renderingcited by 439
  28. 2017Deep Learning for Medical Image Analysiscited by 399
  29. 2019AutoML: A Survey of the State-of-the-Artcited by 388
  30. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
  31. 2018Enhanced-alignment Measure for Binary Foreground Map Evaluationcited by 370
  32. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 368
  33. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
  34. 2021Anabranch Network for Camouflaged Object Segmentationcited by 227
  35. 2021TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentationcited by 225
  36. 2018Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptationcited by 218
  37. 2016Learning Deep Representations of Fine-grained Visual Descriptionscited by 182
  38. 2020Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learningcited by 174

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 16; 2026: 6.

29 papers.

Sort:

By area:

  1. 2024AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoMLcited by 84
  2. 2025ML-Master: Towards AI-for-AI via Integration of Exploration and Reasoningcited by 57
  3. 2025R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Sciencecited by 33
  4. 2025ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineeringcited by 26
  5. 2024SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learningcited by 23
  6. 2024Kolb-Based Experiential Learning for Generalist Agents with Human-Level Kaggle Data Science Performancecited by 22
  7. 2025MLZero: A Multi-Agent System for End-to-end Machine Learning Automationcited by 22
  8. 2025TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fieldscited by 15
  9. 2025I-MCTS: Enhancing Agentic AutoML via Introspective Monte Carlo Tree Searchcited by 14
  10. 2024LLM-based feature generation from text for interpretable machine learningcited by 12
  11. 2026MultiModalPFN: Extending Prior-Data Fitted Networks for Multimodal Tabular Learningcited by 9
  12. 2024Bag of Tricks for Multimodal AutoML with Image, Text, and Tabular Datacited by 6
17 more
  1. 2026TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Explorationcited by 6
  2. 2024Transfer Learning for Finetuning Large Language Modelscited by 5
  3. 2025Kaputt: A Large-Scale Dataset for Visual Defect Detectioncited by 5
  4. 2024Automated Ensemble Multimodal Machine Learning for Healthcarecited by 3
  5. 2025Multimodal Representation Learning and Fusioncited by 3
  6. 2026MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Imagecited by 2
  7. 2025Generative Distribution Prediction: A Unified Approach to Multimodal Learningcited by 1
  8. 2025AutoIntent: AutoML for Text Classificationcited by 1
  9. 2025Towards Cross-Modal Error Detection with Tables and Imagescited by 1
  10. 2025AutoML Algorithms for Online Generalized Additive Model Selection: Application to Electricity Demand Forecastingcited by 0
  11. 2025Quickly Tuning Foundation Models for Image Segmentationcited by 0
  12. 2025ML2B: Benchmarking LLMs on Cross-Lingual ML Pipeline Generationcited by 0
  13. 2025BioAutoML-NAS: An End-to-End AutoML Framework for Multimodal Insect Classification via Neural Architecture Search on Large-Scale Biodiversity Datacited by 0
  14. 2025Cost Savings from Automatic Quality Assessment of Generated Imagescited by 0
  15. 2026SNAC-Pack 2.0: Scaled-Out Surrogate Neural Architecture Codesigncited by 0
  16. 2026Modular Multimodal Classification Without Fine-Tuning: A Simple Compositional Approachcited by 0
  17. 2026Converting Expert Deliberation into Financial Signals Through A Context-Aware NLP Pipelinecited by 0