A ROS 2 Wrapper for Florence-2: Multi-Mode Local Vision-Language Inference for Robotic Systems
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Foundation vision-language models are becoming increasingly relevant to robotics because they can provide richer semantic perception than narrow task-specific pipelines. However, their practical adoption in robot software stacks still depends on reproducible middleware integrations rather than on model quality alone. Florence-2 is especially attractive in this regard because it unifies captioning, optical character recognition, open-vocabulary detection, grounding and related vision-language tasks within a comparatively manageable model size. This article presents a ROS 2 wrapper for Florence-2 that exposes the model through three complementary interaction modes: continuous topic-driven processing, synchronous service calls and asynchronous actions. The wrapper is designed for local execution and supports both native installation and Docker container deployment. It also combines generic JSON outputs with standard ROS 2 message bindings for detection-oriented tasks. A functional validation is reported together with a throughput study on several GPUs, showing that local deployment is feasible with consumer grade hardware. The repository is publicly available here: https://github.com/JEDominguezVidal/florence2_ros2_wrapper
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 27 references that are in the map. First posted on arXiv 1 Apr 2026.
Builds on
20 of 27 references.
Closest ancestors · 2
Same direction, newest first
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul
- 2023PaLM-E: An Embodied Multimodal Language Model6 Mar
Other references · 18
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023Segment Anythingcited by 8,158
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2021Florence: A New Foundation Model for Computer Visioncited by 802
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
- 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
- 2023MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Trainingcited by 96
- 2024Understanding Large-Language Model (LLM)-powered Human-Robot Interactioncited by 43
- 2024SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognitioncited by 35
- 2024ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoningcited by 27
- 2016Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebocited by 18
- 2024Enabling Novel Mission Operations and Interactions with ROSA: The Robot Operating System Agentcited by 16
6 more
- 2023NerfBridge: Bringing Real-time, Online Neural Radiance Field Training to Roboticscited by 9
- 2019gym-gazebo2, a toolkit for reinforcement learning using ROS 2 and Gazebocited by 7
- 2026The human intention. A taxonomy attempt and its applications to roboticscited by 4
- 2026When the Inference Meets the Explicitness or Why Multimodality Can Make Us Forget About the Perfect Predictorcited by 4
- 2023FogROS2-SGC: A ROS2 Cloud Robotics Platform for Secure Global Connectivitycited by 3
- 2023Wrapyfi: A Python Wrapper for Integrating Robots, Sensors, and Applications across Multiple Middlewarecited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Fine-Tuning Florence2 for Enhanced Object Detection in Un-constructed Environments: Vision-Language Model Approach
- 2023RoboLLM: Robotic Vision Tasks Grounded on Multimodal Large Language Models
- 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
- 2026Efficient Vision-Language-Action Management and Serving for Robot Factories
- 2026ROS2SmolVLA: Enabling Small Vision-Language-Action Models for Integration into Industrial-Grade Lightweight Robots
- 2026LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal Navigation
- 20253D-Grounded Vision-Language Framework for Robotic Task Planning: Automated Prompt Synthesis and Supervised Reasoning
- 2026Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment