Efficient Visual Pointing for Embodied AI:Agent-Driven Data Synthesis, Cross-Block Attention, and Iterative Correction
Not on the map. Why?
Abstract (arXiv)
Visual pointing maps a language instruction to pixel co ordinates, a core skill for embodied AI. We describe our PointArena 2026 solution, which achieves 77.2% overall accuracy and ranks second on the benchmark. The ap proach targets three failure modes. First, agent-driven syn thesis builds large semantic and anchor-relative candidate pools; the server inventory contains 55,372 processed out puts, 53,772 de-duplicated sample IDs, and 37,574 train able completed or accepted rows. Second, a determinis tic steerable-data pipeline creates a verified 10,000-sample main set, plus reserve samples, using masks, templates, and path verification. Third, two model-side modules address complementary errors: AttnRes adds gated cross-block at tention for steerability, while ABC correction encodes per turbed coordinates with visual features for general coordi nate grounding. Category-aware routing combines comple mentary specialists; local validation used to select experts records 93.9% Affordance, 82.6% Spatial Relation, 78.2% Reasoning, 70.4% Counting, and 63.0% Steerability.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 14 references that are in the map. First posted on arXiv 29 Jun 2026.
Builds on
11 of 14 references.
References · 11
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Segment Anythingcited by 8,234
- 2023Visual Instruction Tuningcited by 7,869
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
- 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
- 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
- 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 329
- 2025PointArena: Probing Multimodal Grounding Through Language-Guided Pointingcited by 31
Built on by
Not cited yet.