mlsanityData to 14 Oct 2026

Lang2Lift: A Language-Guided Autonomous Forklift System for Outdoor Industrial Pallet Handling

arXiv 2508.15427 ↗First posted 21 Aug 2025cs.RO cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Automating pallet handling in outdoor logistics and construction environments remains challenging due to unstructured scenes, variable pallet configurations, and changing environmental conditions. In this paper, we present Lang2Lift, an end-to-end language-guided autonomous forklift system designed to support practical pallet pick-up operations in real-world outdoor settings. The system enables operators to specify target pallets using natural language instructions, allowing flexible selection among multiple pallets with different loads and spatial arrangements. Lang2Lift integrates foundation-model-based perception modules with motion planning and control in a closed-loop autonomy pipeline. Language-grounded visual perception is used to identify and segment target pallets, followed by 6D pose estimation and geometric refinement to generate manipulation-feasible insertion poses. The resulting pose estimates are directly coupled with the forklift planning and control modules to execute fully autonomous pallet pick-up maneuvers. We deploy and evaluate the proposed system on the ADAPT autonomous outdoor forklift platform across diverse real-world scenarios, including cluttered scenes, variable lighting, and different payload configurations. Tolerance-based pose evaluation further indicates accuracy sufficient for successful fork insertion. Timing and failure analyses highlight key deployment trade-offs and practical limitations, providing insights into integrating language-guided perception within industrial automation systems. Video demonstrations are available at https://eric-nguyen1402.github.io/lang2lift.github.io/

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 38 references that are in the map. First posted on arXiv 21 Aug 2025.

Builds on

16 of 38 references.

References · 16

  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  2. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  3. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  4. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
  5. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 429
  6. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
  7. 2022MegaPose: 6D Pose Estimation of Novel Objects via Render & Comparecited by 201
  8. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 148
  9. 2021FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimationcited by 144
  10. 2022FS6D: Few-Shot 6D Pose Estimation of Novel Objectscited by 81
  11. 2024Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactivelycited by 75
  12. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
4 more
  1. 2018Detection, localisation and tracking of pallets using machine learning techniques and 2D range datacited by 10
  2. 2019Neural Semantic Parsing with Anonymization for Command Understanding in General-Purpose Service Robotscited by 4
  3. 2023Natural Language Instructions for Intuitive Human Interaction with Robotic Assistants in Field Construction Workcited by 3
  4. 2025ADAPT: An Autonomous Forklift for Construction Site Operationcited by 3

Built on by

Not cited yet.