mlsanityData to 14 Oct 2026

Fly-by-Code: Embodied Coding Agents for Aerial Manipulation with Active Visual and Physical Feedback

arXiv 2610.12549 ↗First posted 8 Oct 2026cs.RO cs.CV cs.LGcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Embodied coding agents construct robot policies from natural-language instructions by generating executable programs and refining them from execution feedback. This refinement assumes that execution feedback reveals whether the task succeeded and, after a failure, provides sufficient evidence to guide program revision. Aerial manipulation violates this assumption in two ways. Payload constraints restrict onboard camera coverage and leave task-critical outcomes outside the robot's view (partial observability), while strong dynamic coupling and limited control margins make physical interactions fragile (limited interaction robustness) and leave failure causes unresolved. Re-examining the same feedback cannot recover observations that were never collected or separate failure causes that the available evidence does not distinguish. To acquire this missing evidence, we introduce Active Feedback (AF), an active "View and Probe" framework. When execution feedback leaves unresolved questions, the agent synthesizes a separate diagnostic program that runs in simulation to inspect unseen outcomes from informative viewpoints (View) or test hypotheses about failure causes through targeted physical interactions (Probe). Across six simulated aerial manipulation tasks with 60 trials per condition, AF reduces false completion declarations from 25 to 5 and raises final task success from 31.7% to 75% compared with terminal feedback under the same coding-agent configuration and task-execution limit. Using shared robot APIs, task programs refined with AF in simulation run on a physical aerial manipulator without task-program source-code modification and complete multi-stage manipulation tasks in the real world. These results show that executable diagnosis fills gaps in execution feedback for embodied coding agents, improving both completion assessment and program refinement in aerial manipulation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 37 of its 53 references that are in the map. First posted on arXiv 8 Oct 2026.

Builds on

37 of 53 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026ASPIRE: Agentic /Skills Discovery for Robotics30 Jun
  2. 2026ENPIRE: Agentic Robot Policy Self-Improvement in the Real World18 Jun
  3. 2026CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation23 Mar

Other references · 34

  1. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  2. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  3. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
  4. 2023Scaling Open-Vocabulary Object Detectioncited by 343
  5. 2025SAM 3D: 3Dfy Anything in Imagescited by 247
  6. 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correctioncited by 175
  7. 2016Revisiting Active Perceptioncited by 94
  8. 2016Interactive Perception: Leveraging Action in Perception and Perception in Actioncited by 78
  9. 2024RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesiscited by 58
  10. 2023Chat with the Environment: Interactive Multimodal Perception Using Large Language Modelscited by 43
  11. 2025Vision in Action: Learning Active Perception from Human Demonstrationscited by 37
  12. 2025PyRoki: A Modular Toolkit for Robot Kinematic Optimizationcited by 30
22 more
  1. 2024RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulationcited by 26
  2. 2026ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulationcited by 18
  3. 2025UMI-on-Air: Embodiment-Aware Guidance for Embodiment-Agnostic Visuomotor Policiescited by 11
  4. 2025Flying Hand: End-Effector-Centric Framework for Versatile Aerial Manipulation Teleoperation and Policy Learningcited by 9
  5. 2024Flying Calligrapher: Contact-Aware Motion and Force Planning and Control for Aerial Manipulationcited by 8
  6. 2026SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Roboticscited by 8
  7. 2026π, But Make It Fly: Physics-Guided Transfer of VLA Models to Aerial Manipulationcited by 7
  8. 2026Boxer: Robust Lifting of Open-World 2D Bounding Boxes to 3Dcited by 7
  9. 2024Observe Then Act: Asynchronous Active Vision-Action Model for Robotic Manipulationcited by 5
  10. 2025AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Modelscited by 4
  11. 2026PhysCaP: Grounding Code-as-Policy Agent with Physics-Informed Explorationcited by 4
  12. 2013Geometric Nonlinear PID Control of a Quadrotor UAV on SE(3)cited by 3
  13. 2024Interactive Perception for Deformable Object Manipulationcited by 3
  14. 2024An Open-Source Soft Robotic Platform for Autonomous Aerial Manipulation in the Wildcited by 3
  15. 2024Modular Adaptive Aerial Manipulation under Unknown Dynamic Coupling Forcescited by 3
  16. 2025Whole-Body Integrated Motion Planning for Aerial Manipulatorscited by 3
  17. 2026Confidence-Based Mesh Extraction from 3D Gaussianscited by 3
  18. 2026A Few Words Go a Long Way: Language Guided Robot Policy Synthesiscited by 3
  19. 2025Autonomous Aerial Manipulation at Arbitrary Pose in SE(3) with Robust Control and Whole-body Planningcited by 2
  20. 2026AeroGrab: A Unified Framework for Aerial Grasping in Cluttered Environmentscited by 2
  21. 2026AM-Bench: A Modular Simulation Suite and Benchmark for Aerial Manipulation Policy Learningcited by 2
  22. 2024Creating and Repairing Robot Programs in Open-World Domainscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026EmbodiedEvo: Self-Evolving Coding Agent For Physical Intelligence
  2. 2026CodeActionBench: Evaluating Agentic Code-as-Policy for Embodied Manipulation
  3. 2026ActiveFly-Bench: Aligning Embodied Question Answering with Vision-Language-Action for Aerial Embodied Perception
  4. 2025Drones that Think on their Feet: Sudden Landing Decisions with Embodied AI
  5. 2026From Passive Execution to Active Exploration: Agentic Embodied Manipulation in Realistic Environments
  6. 2026From Local Whole-Body VLA Behaviors to Scene-Scale Aerial Manipulation
  7. 2026Encore: Few-Shot Agentic Discovery of Manipulation Strategies
  8. 2026Embodied Skill Bench: Benchmarking How Well Agents with Embodied Skills Perform on Physical Tasks