mlsanityData to 7 Oct 2026

GAgent: An Adaptive Rigid-Soft Gripping Agent with Vision Language Models for Complex Lighting Environments

arXiv 2403.10850 ↗First posted 16 Mar 2024cs.RO cs.AIcited by 6

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

This paper introduces GAgent: an Gripping Agent designed for open-world environments that provides advanced cognitive abilities via VLM agents and flexible grasping abilities with variable stiffness soft grippers. GAgent comprises three primary components - Prompt Engineer module, Visual-Language Model (VLM) core and Workflow module. These three modules enhance gripper success rates by recognizing objects and materials and accurately estimating grasp area even under challenging lighting conditions. As part of creativity, researchers also created a bionic hybrid soft gripper with variable stiffness capable of gripping heavy loads while still gently engaging objects. This intelligent agent, featuring VLM-based cognitive processing with bionic design, shows promise as it could potentially benefit UAVs in various scenarios.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 11 of its 35 references that are in the map. First posted on arXiv 16 Mar 2024.

Builds on

11 of 35 references.

References · 11

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  5. 2022Emergent Abilities of Large Language Modelscited by 2,483
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  8. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  9. 2016Team Delft's Robot Winner of the Amazon Picking Challenge 2016cited by 10
  10. 2023SCRNet: a Retinex Structure-based Low-light Enhancement Model Guided by Spatial Consistencycited by 8
  11. 2023Human Demonstrations are Generalizable Knowledge for Robotscited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 5.

6 papers.

  1. 2025TSCnet: A Text-driven Semantic-level Controllable Framework for Customized Low-Light Image Enhancementcited by 9
  2. 2024FALCON: Feedback-driven Adaptive Long/short-term memory reinforced Coding Optimization systemcited by 8
  3. 2025PromptLNet: Region-Adaptive Aesthetic Enhancement via Prompt Guidance in Low-Light Enhancement Netcited by 8
  4. 2025A Cascading Cooperative Multi-agent Framework for On-ramp Merging Control Integrating Large Language Modelscited by 5
  5. 2025MARS: Memory-Enhanced Agents with Reflective Self-improvementcited by 2
  6. 2025TDRI: Two-Phase Dialogue Refinement and Co-Adaptation for Interactive Image Generationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Using Visual Language Models to Control Bionic Hands: Assessment of Object Perception and Grasp Inference
  2. 2026Agentic RAG-VLM: Affordance-Aware Retrieval-Augmented Generation with Self-Reflective Planning for Robotic Grasping
  3. 2025VacuumVLA: Boosting VLA Capabilities via a Unified Suction and Gripping Tool for Complex Robotic Manipulation
  4. 2025RoboDexVLM: Visual Language Model-Enabled Task Planning and Motion Control for Dexterous Robot Manipulation
  5. 2026VILAS: A VLA-Integrated Low-cost Architecture with Soft Grasping for Robotic Manipulation
  6. 2025Tactile-VLA: Unlocking Vision-Language-Action Model's Physical Knowledge for Tactile Generalization
  7. 2025VLMgineer: Vision Language Models as Robotic Toolsmiths
  8. 2026AIR-VLA: Vision-Language-Action Systems for Aerial Manipulation