mlsanityData to 14 Oct 2026

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

arXiv 2311.06242 ↗First posted 10 Nov 2023cs.CVcited by 410

Not on the map. Why?

Cited by 95 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce Florence-2, a novel vision foundation model with a unified, prompt-based representation for a variety of computer vision and vision-language tasks. While existing large vision models excel in transfer learning, they struggle to perform a diversity of tasks with simple instructions, a capability that implies handling the complexity of various spatial hierarchy and semantic granularity. Florence-2 was designed to take text-prompt as task instructions and generate desirable results in text forms, whether it be captioning, object detection, grounding or segmentation. This multi-task learning setup demands large-scale, high-quality annotated data. To this end, we co-developed FLD-5B that consists of 5.4 billion comprehensive visual annotations on 126 million images, using an iterative strategy of automated image annotation and model refinement. We adopted a sequence-to-sequence structure to train Florence-2 to perform versatile and comprehensive vision tasks. Extensive evaluations on numerous tasks demonstrated Florence-2 to be a strong vision foundation model contender with unprecedented zero-shot and fine-tuning capabilities.

Key facts. Cited by 410 papers in the map (data through 14 Oct 2026). Builds on 73 of its 97 references that are in the map. First posted on arXiv 10 Nov 2023.

Builds on

73 of 97 references.

References · 73

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  9. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  10. 2023Segment Anythingcited by 8,234
  11. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  12. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
38 more
  1. 2020End-to-End Object Detection with Transformerscited by 7,414
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2017Mask R-CNNcited by 7,277
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  9. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  12. 2022A ConvNet for the 2020scited by 3,579
  13. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  14. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  15. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  16. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  17. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  18. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  19. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  20. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  21. 2019Towards VQA Models That Can Readcited by 1,718
  22. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  23. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  24. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  25. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  26. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  27. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  28. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  29. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  30. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  31. 2016Modeling Context in Referring Expressionscited by 1,063
  32. 2021Scaling Vision Transformerscited by 999
  33. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
  34. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 919
  35. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  36. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  37. 2021Florence: A New Foundation Model for Computer Visioncited by 804
  38. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 54; 2025: 201; 2026: 151.

410 papers.

Sort:

By area:

  1. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  2. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  3. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 329
  4. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 303
  5. 2024Building and better understanding vision-language models: insights and future directionscited by 173
  6. 2024Large Language Model-Brained GUI Agents: A Surveycited by 162
  7. 2023GSVA: Generalized Segmentation via Multimodal Large Language Modelscited by 154
  8. 2024Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Modelscited by 116
  9. 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 106
  10. 2025HD-EPIC: A Highly-Detailed Egocentric Video Datasetcited by 104
  11. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 96
  12. 2025Latent Diffusion Model without Variational Autoencodercited by 88
38 more
  1. 2025HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generationcited by 87
  2. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  3. 2024Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvementcited by 74
  4. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  5. 2024Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Datacited by 67
  6. 2025Any2AnyTryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Taskscited by 63
  7. 2025FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policiescited by 57
  8. 2025XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulationcited by 56
  9. 2024LLM-Seg: Bridging Image Segmentation and Large Language Model Reasoningcited by 52
  10. 2024GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Taskscited by 51
  11. 2025RoboGround: Robotic Manipulation with Grounded Vision-Language Priorscited by 51
  12. 2024V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Modelscited by 47
  13. 2025SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulationcited by 46
  14. 2025LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformercited by 43
  15. 2025UFO2: The Desktop AgentOScited by 41
  16. 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaborationcited by 41
  17. 2024Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation datacited by 39
  18. 2024MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMscited by 39
  19. 2025VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Modelscited by 36
  20. 2025Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Modelcited by 36
  21. 2024LocCa: Visual Pretraining with Location-aware Captionerscited by 35
  22. 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
  23. 2024Contrastive Localized Language-Image Pre-Trainingcited by 33
  24. 2024ChatRex: Taming Multimodal LLM for Joint Perception and Understandingcited by 30
  25. 2024Schedule On the Fly: Diffusion Time Prediction for Faster and Better Image Generationcited by 30
  26. 2025RzenEmbed: Towards Comprehensive Multimodal Retrievalcited by 29
  27. 2025LoRA-Edit: Controllable First-Frame-Guided Video Editing via Mask-Aware LoRA Fine-Tuningcited by 28
  28. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 28
  29. 2025BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learningcited by 27
  30. 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Roboticscited by 27
  31. 2025DynamicEarth: How Far are We from Open-Vocabulary Change Detection?cited by 21
  32. 2025ModernVBERT: Towards Smaller Visual Document Retrieverscited by 21
  33. 2024Boosting Latent Diffusion with Perceptual Objectivescited by 20
  34. 2024Interpreting Object-level Foundation Models via Visual Precision Searchcited by 20
  35. 2025InterActHuman: Multi-Concept Human Animation with Layout-Aligned Audio Conditionscited by 20
  36. 2025DexVLG: Dexterous Vision-Language-Grasp Model at Scalecited by 20
  37. 2025Scaling Pre-training to One Hundred Billion Data for Vision Language Modelscited by 19
  38. 2025IntrinsiX: High-Quality PBR Generation using Image Priorscited by 19

50 most cited of 410 All 410 (JSON)