mlsanityData to 7 Oct 2026

Active Fine-Tuning of Multi-Task Policies

arXiv 2410.05026 ↗First posted 7 Oct 2024cs.LG cs.ROcited by 8

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Pre-trained generalist policies are rapidly gaining relevance in robot learning due to their promise of fast adaptation to novel, in-domain tasks. This adaptation often relies on collecting new demonstrations for a specific task of interest and applying imitation learning algorithms, such as behavioral cloning. However, as soon as several tasks need to be learned, we must decide which tasks should be demonstrated and how often? We study this multi-task problem and explore an interactive framework in which the agent adaptively selects the tasks to be demonstrated. We propose AMF (Active Multi-task Fine-tuning), an algorithm to maximize multi-task policy performance under a limited demonstration budget by collecting demonstrations yielding the largest information gain on the expert policy. We derive performance guarantees for AMF under regularity assumptions and demonstrate its empirical effectiveness to efficiently fine-tune neural policies in complex and high-dimensional environments.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 63 of its 89 references that are in the map. First posted on arXiv 7 Oct 2024.

Builds on

63 of 89 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Data Quality in Imitation Learning4 Jun
  2. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr

Other references · 61

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2016Layer Normalizationcited by 5,579
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  8. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  9. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  11. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  12. 2023Let's Verify Step by Stepcited by 3,598
38 more
  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  7. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  8. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  9. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  10. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  11. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  12. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  13. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  14. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  15. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  16. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  17. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  18. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  19. 2023Understanding plasticity in neural networkscited by 170
  20. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  21. 2019A General Framework for Uncertainty Estimation in Deep Learningcited by 119
  22. 2023RoboCat: A Self-Improving Generalist Agent for Robotic Manipulationcited by 108
  23. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 104
  24. 2020Error Bounds of Imitating Policies and Environmentscited by 90
  25. 2019Approximate Inference Turns Deep Networks into Gaussian Processescited by 89
  26. 2021SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarioscited by 83
  27. 2021Rehearsal revealed: The limits and merits of revisiting samples in continual learningcited by 76
  28. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 75
  29. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learningcited by 74
  30. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shiftcited by 68
  31. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  32. 2023Prediction-Oriented Bayesian Active Learningcited by 60
  33. 2017Dynamic Evaluation of Neural Sequence Modelscited by 53
  34. 2021PRISM: A Rich Class of Parameterized Submodular Information Measures for Guided Subset Selectioncited by 50
  35. 2022A Framework and Benchmark for Deep Batch Active Learning for Regressioncited by 47
  36. 2018Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgettingcited by 46
  37. 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Expertscited by 45
  38. 2023RoboHive: A Unified Framework for Robot Learningcited by 42

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 2.

8 papers.

  1. 2025Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Mergingcited by 19
  2. 2025DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learningcited by 11
  3. 2025Local Mixtures of Experts: Essentially Free Test-Time Training via Model Mergingcited by 9
  4. 2025Test-time Offline Reinforcement Learning on Goal-related Experiencecited by 8
  5. 2025Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learningcited by 8
  6. 2025Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Modelscited by 5
  7. 2026Uncertainty Quantification for Flow-Based Vision-Language-Action Modelscited by 4
  8. 2026From Generalist to Specialist Representationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvement
  2. 2024Effective Tuning Strategies for Generalist Robot Manipulation Policies
  3. 2025Data-Efficient Multitask DAgger
  4. 2025Flexible Multitask Learning with Factorized Diffusion Policy
  5. 2025FDPP: Fine-tune Diffusion Policy with Human Preference
  6. 2023Comparing the Efficacy of Fine-Tuning and Meta-Learning for Few-Shot Policy Imitation
  7. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning
  8. 2022Modular Adaptive Policy Selection for Multi-Task Imitation Learning through Task Division