# Fly-by-Code: Embodied Coding Agents for Aerial Manipulation with Active Visual and Physical Feedback arXiv: https://arxiv.org/abs/2610.12549 · HTML: https://mlsanity.com/paper/2610.12549/ ## Key facts - No citing papers in the map yet (data through 14 Oct 2026). - Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. - Builds on 37 of its 53 references that are in the map. - First posted on arXiv 8 Oct 2026. ## Placed in - LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/) ## Builds on (most cited first, 30 of 37) - [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022) - [ProgPrompt: Generating Situated Robot Task Plans using Large Language Models](https://mlsanity.com/paper/2209.11302/) (2022) - [Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation](https://mlsanity.com/paper/2404.15506/) (2024) - [Scaling Open-Vocabulary Object Detection](https://mlsanity.com/paper/2306.09683/) (2023) - [SAM 3D: 3Dfy Anything in Images](https://mlsanity.com/paper/2511.16624/) (2025) - [REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction](https://mlsanity.com/paper/2306.15724/) (2023) - [Revisiting Active Perception](https://mlsanity.com/paper/1603.02729/) (2016) - [CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation](https://mlsanity.com/paper/2603.22435/) (2026) - [Interactive Perception: Leveraging Action in Perception and Perception in Action](https://mlsanity.com/paper/1604.03670/) (2016) - [RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis](https://mlsanity.com/paper/2402.16117/) (2024) - [ASPIRE: Agentic /Skills Discovery for Robotics](https://mlsanity.com/paper/2607.00272/) (2026) - [Chat with the Environment: Interactive Multimodal Perception Using Large Language Models](https://mlsanity.com/paper/2303.08268/) (2023) - [ENPIRE: Agentic Robot Policy Self-Improvement in the Real World](https://mlsanity.com/paper/2606.19980/) (2026) - [Vision in Action: Learning Active Perception from Human Demonstrations](https://mlsanity.com/paper/2506.15666/) (2025) - [PyRoki: A Modular Toolkit for Robot Kinematic Optimization](https://mlsanity.com/paper/2505.03728/) (2025) - [RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation](https://mlsanity.com/paper/2402.14623/) (2024) - [ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulation](https://mlsanity.com/paper/2601.08325/) (2026) - [UMI-on-Air: Embodiment-Aware Guidance for Embodiment-Agnostic Visuomotor Policies](https://mlsanity.com/paper/2510.02614/) (2025) - [Flying Hand: End-Effector-Centric Framework for Versatile Aerial Manipulation Teleoperation and Policy Learning](https://mlsanity.com/paper/2504.10334/) (2025) - [Flying Calligrapher: Contact-Aware Motion and Force Planning and Control for Aerial Manipulation](https://mlsanity.com/paper/2407.05587/) (2024) - [SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Robotics](https://mlsanity.com/paper/2603.12193/) (2026) - [π, But Make It Fly: Physics-Guided Transfer of VLA Models to Aerial Manipulation](https://mlsanity.com/paper/2603.25038/) (2026) - [Boxer: Robust Lifting of Open-World 2D Bounding Boxes to 3D](https://mlsanity.com/paper/2604.05212/) (2026) - [Observe Then Act: Asynchronous Active Vision-Action Model for Robotic Manipulation](https://mlsanity.com/paper/2409.14891/) (2024) - [AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Models](https://mlsanity.com/paper/2511.01472/) (2025) - [PhysCaP: Grounding Code-as-Policy Agent with Physics-Informed Exploration](https://mlsanity.com/paper/2608.21031/) (2026) - [Geometric Nonlinear PID Control of a Quadrotor UAV on SE(3)](https://mlsanity.com/paper/1304.6765/) (2013) - [Interactive Perception for Deformable Object Manipulation](https://mlsanity.com/paper/2403.05177/) (2024) - [An Open-Source Soft Robotic Platform for Autonomous Aerial Manipulation in the Wild](https://mlsanity.com/paper/2409.07662/) (2024) - [Modular Adaptive Aerial Manipulation under Unknown Dynamic Coupling Forces](https://mlsanity.com/paper/2410.08285/) (2024) ## Abstract (arXiv) Embodied coding agents construct robot policies from natural-language instructions by generating executable programs and refining them from execution feedback. This refinement assumes that execution feedback reveals whether the task succeeded and, after a failure, provides sufficient evidence to guide program revision. Aerial manipulation violates this assumption in two ways. Payload constraints restrict onboard camera coverage and leave task-critical outcomes outside the robot's view (partial observability), while strong dynamic coupling and limited control margins make physical interactions fragile (limited interaction robustness) and leave failure causes unresolved. Re-examining the same feedback cannot recover observations that were never collected or separate failure causes that the available evidence does not distinguish. To acquire this missing evidence, we introduce Active Feedback (AF), an active "View and Probe" framework. When execution feedback leaves unresolved questions, the agent synthesizes a separate diagnostic program that runs in simulation to inspect unseen outcomes from informative viewpoints (View) or test hypotheses about failure causes through targeted physical interactions (Probe). Across six simulated aerial manipulation tasks with 60 trials per condition, AF reduces false completion declarations from 25 to 5 and raises final task success from 31.7% to 75% compared with terminal feedback under the same coding-agent configuration and task-execution limit. Using shared robot APIs, task programs refined with AF in simulation run on a physical aerial manipulator without task-program source-code modification and complete multi-stage manipulation tasks in the real world. These results show that executable diagnosis fills gaps in execution feedback for embodied coding agents, improving both completion assessment and program refinement in aerial manipulation.