mlsanityData to 14 Oct 2026

Language-Conditioned Representations and Mixture-of-Experts Policy for Robust Multi-Task Robotic Manipulation

arXiv 2510.24055 ↗First posted 28 Oct 2025cs.RO cs.LGcited by 2

On the map: Mixture-of-experts → MoE in reinforcement learning and robotics

also in 1 more
Abstract (arXiv)

Perceptual ambiguity and task conflict limit multitask robotic manipulation via imitation learning. We propose a framework combining a Language-Conditioned Visual Representation (LCVR) module and a Language-conditioned Mixture-ofExperts Density Policy (LMoE-DP). LCVR resolves perceptual ambiguities by grounding visual features with language instructions, enabling differentiation between visually similar tasks. To mitigate task conflict, LMoE-DP uses a sparse expert architecture to specialize in distinct, multimodal action distributions, stabilized by gradient modulation. On real-robot benchmarks, LCVR boosts Action Chunking with Transformers (ACT) and Diffusion Policy (DP) success rates by 33.75% and 25%, respectively. The full framework achieves a 79% average success, outperforming the advanced baseline by 21%. Our work shows that combining semantic grounding and expert specialization enables robust, efficient multi-task manipulation

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in MoE in reinforcement learning and robotics, one of 12 directions in Mixture-of-experts. Builds on 37 of its 48 references that are in the map. First posted on arXiv 28 Oct 2025.

Builds on

37 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning1 Jul

Other references · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  8. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  9. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  11. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  12. 2019Root Mean Square Layer Normalizationcited by 1,480
24 more
  1. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  2. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  3. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  4. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  5. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  6. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  7. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
  8. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  9. 2021Implicit Behavioral Cloningcited by 452
  10. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 375
  11. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 282
  12. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 225
  13. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 195
  14. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 137
  15. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 122
  16. 2025OpenHelix: A Short Survey, Empirical Analysis, and Open-Source Dual-System VLA Model for Robotic Manipulationcited by 82
  17. 2023FAMO: Fast Adaptive Multitask Optimizationcited by 78
  18. 2024EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learningcited by 75
  19. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 74
  20. 2024ManiCM: Real-time 3D Diffusion Policy via Consistency Model for Robotic Manipulationcited by 57
  21. 2025Hume: Introducing System-2 Thinking in Visual-Language-Action Modelcited by 56
  22. 2020A Causal View on Robustness of Neural Networkscited by 53
  23. 2024Motion Before Action: Diffusing Object Motion as Manipulation Conditioncited by 16
  24. 2024Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulationcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026SARM2: Multi-Task Stage Aware Reward Modeling for Self Improving Robotic Manipulationcited by 2
  2. 2026Trace-Focused Diffusion Policy for Multi-Modal Action Disambiguation in Long-Horizon Robotic Manipulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Semantically Structured Mixture-of-Experts for Compositional Robotic Manipulation ↗
  2. 2026Decoupling Semantics and Geometric Grounding: Spatial Visual Prompts for Language-Conditioned Imitation Learning
  3. 2023Programmatically Grounded, Compositionally Generalizable Robotic Manipulation
  4. 2025CodeDiffuser: Attention-Enhanced Diffusion Policy via VLM-Generated Code for Instruction Ambiguity
  5. 2025Learning Compositional Behaviors from Demonstration and Language
  6. 2024Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitation
  7. 2026Abstracting Robot Manipulation Skills via Mixture-of-Experts Diffusion Policies
  8. 2026Vision-Language Grounded Task-Context-Aware Imitation Learning for Robotic Disassembly