# A ROS 2 Wrapper for Florence-2: Multi-Mode Local Vision-Language Inference for Robotic Systems

arXiv: https://arxiv.org/abs/2604.01179 · HTML: https://mlsanity.com/paper/2604.01179/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 20 of its 27 references that are in the map.
- First posted on arXiv 1 Apr 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [Florence: A New Foundation Model for Computer Vision](https://mlsanity.com/paper/2111.11432/) (2021)
- [VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models](https://mlsanity.com/paper/2307.05973/) (2023)
- [Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks](https://mlsanity.com/paper/2311.06242/) (2023)
- [MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training](https://mlsanity.com/paper/2311.17049/) (2023)
- [Understanding Large-Language Model (LLM)-powered Human-Robot Interaction](https://mlsanity.com/paper/2401.03217/) (2024)
- [SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognition](https://mlsanity.com/paper/2410.10624/) (2024)
- [ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning](https://mlsanity.com/paper/2406.19741/) (2024)
- [Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebo](https://mlsanity.com/paper/1608.05742/) (2016)
- [Enabling Novel Mission Operations and Interactions with ROSA: The Robot Operating System Agent](https://mlsanity.com/paper/2410.06472/) (2024)
- [NerfBridge: Bringing Real-time, Online Neural Radiance Field Training to Robotics](https://mlsanity.com/paper/2305.09761/) (2023)
- [gym-gazebo2, a toolkit for reinforcement learning using ROS 2 and Gazebo](https://mlsanity.com/paper/1903.06278/) (2019)
- [The human intention. A taxonomy attempt and its applications to robotics](https://mlsanity.com/paper/2602.15963/) (2026)
- [When the Inference Meets the Explicitness or Why Multimodality Can Make Us Forget About the Perfect Predictor](https://mlsanity.com/paper/2602.18850/) (2026)
- [FogROS2-SGC: A ROS2 Cloud Robotics Platform for Secure Global Connectivity](https://mlsanity.com/paper/2306.17157/) (2023)
- [Wrapyfi: A Python Wrapper for Integrating Robots, Sensors, and Applications across Multiple Middleware](https://mlsanity.com/paper/2302.09648/) (2023)

## Built on by (most cited first, 1 of 1)

- [From Technical Metrics to User Perception: A User Study of a Multimodal Human-Robot Interaction System for Object Detection and Grasping](https://mlsanity.com/paper/2607.00530/) (2026)

## Abstract (arXiv)

Foundation vision-language models are becoming increasingly relevant to robotics because they can provide richer semantic perception than narrow task-specific pipelines. However, their practical adoption in robot software stacks still depends on reproducible middleware integrations rather than on model quality alone. Florence-2 is especially attractive in this regard because it unifies captioning, optical character recognition, open-vocabulary detection, grounding and related vision-language tasks within a comparatively manageable model size. This article presents a ROS 2 wrapper for Florence-2 that exposes the model through three complementary interaction modes: continuous topic-driven processing, synchronous service calls and asynchronous actions. The wrapper is designed for local execution and supports both native installation and Docker container deployment. It also combines generic JSON outputs with standard ROS 2 message bindings for detection-oriented tasks. A functional validation is reported together with a throughput study on several GPUs, showing that local deployment is feasible with consumer grade hardware. The repository is publicly available here: https://github.com/JEDominguezVidal/florence2_ros2_wrapper