mlsanityData to 14 Oct 2026

CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulation

arXiv 2506.09343 ↗First posted 11 Jun 2025cs.CV cs.ROcited by 7

Not on the map. Why?

Abstract (arXiv)

Correct use of electrical appliances has significantly improved human life quality. Unlike simple tools that can be manipulated with common sense, different parts of electrical appliances have specific functions defined by manufacturers. If we want the robot to heat bread by microwave, we should enable them to review the microwave manual first. From the manual, it can learn about component functions, interaction methods, and representative task steps about appliances. However, previous manual-related works remain limited to question-answering tasks while existing manipulation researchers ignore the manual's important role and fail to comprehend multi-page manuals. In this paper, we propose the first manual-based appliance manipulation benchmark CheckManual. Specifically, we design a large model-assisted human-revised data generation pipeline to create manuals based on CAD appliance models. With these manuals, we establish novel manual-based manipulation challenges, metrics, and simulator environments for model performance evaluation. Furthermore, we propose the first manual-based manipulation planning model ManualPlan to set up a group of baselines for the CheckManual benchmark.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 29 of its 34 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

29 of 34 references.

References · 29

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Segment Anythingcited by 8,234
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  4. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  6. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  7. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  8. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
  9. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  10. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 429
  11. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 349
  12. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 315
17 more
  1. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 266
  2. 2022AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domainscited by 208
  3. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 189
  4. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 162
  5. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 157
  6. 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 133
  7. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
  8. 2024CoPa: General Robotic Manipulation through Spatial Constraints of Parts with Foundation Modelscited by 99
  9. 2023LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understandingcited by 95
  10. 2021VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objectscited by 80
  11. 2023Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skillcited by 77
  12. 2025OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraintscited by 59
  13. 2024Omni6DPose: A Benchmark and Model for Universal 6D Object Pose Estimation and Trackingcited by 50
  14. 2023GenPose: Generative Category-level Object Pose Estimation via Diffusion Modelscited by 48
  15. 2021Question Answering over Electronic Devices: A New Benchmark Dataset and a Multi-Task Learning based QA Frameworkcited by 21
  16. 2023SARA-RT: Scaling up Robotics Transformers with Self-Adaptive Robust Attentioncited by 20
  17. 2023MPMQA: Multimodal Question Answering on Product Manualscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.

  1. 2026Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Modelscited by 4
  2. 2025RealAppliance: Let High-fidelity Appliance Assets Controllable and Workable as Aligned Real Manualscited by 2
  3. 2026Anticipation-VLA: Solving Long-Horizon Embodied Tasks via Anticipation-based Subgoal Generationcited by 2
  4. 2026AssemblyBench: Physics-Aware Assembly of Complex Industrial Objectscited by 2
  5. 2026Lost in Instructions: Study of Blind Users' Experiences with DIY Manuals and AI-Rewritten Instructions for Assembly, Operation, and Troubleshooting of Tangible Productscited by 0
  6. 2026KD-Judge: A Knowledge-Driven Automated Judge Framework for Functional Fitness Movements on Edge Devicescited by 0
  7. 2026Scaling Manual-Grounded Appliance Manipulation with Data Synthesis and Unified Planningcited by 0