# Curriculum Learning with Hindsight Experience Replay for Sequential Object Manipulation Tasks

arXiv: https://arxiv.org/abs/2008.09377 · HTML: https://mlsanity.com/paper/2008.09377/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Aug 2020.

## Built on by (most cited first, 10 of 10)

- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Boost Your Human Image Generation Model via Direct Preference Optimization](https://mlsanity.com/paper/2405.20216/) (2024)
- [Relay Hindsight Experience Replay: Self-Guided Continual Reinforcement Learning for Sequential Object Manipulation Tasks with Sparse Rewards](https://mlsanity.com/paper/2208.00843/) (2022)
- [An Open-Source Multi-Goal Reinforcement Learning Environment for Robotic Manipulation with Pybullet](https://mlsanity.com/paper/2105.05985/) (2021)
- [A Psychology-based Unified Dynamic Framework for Curriculum Learning](https://mlsanity.com/paper/2408.05326/) (2024)
- [Cold-start Bundle Recommendation via Popularity-based Coalescence and Curriculum Heating](https://mlsanity.com/paper/2310.03813/) (2023)
- [MRHER: Model-based Relay Hindsight Experience Replay for Sequential Object Manipulation Tasks with Sparse Rewards](https://mlsanity.com/paper/2306.16061/) (2023)
- [Curricular Transfer Learning for Sentence Encoded Tasks](https://mlsanity.com/paper/2308.01849/) (2023)
- [CL4KGE: A Curriculum Learning Method for Knowledge Graph Embedding](https://mlsanity.com/paper/2408.14840/) (2024)
- [PMG: Progressive Motion Generation via Sparse Anchor Postures Curriculum Learning](https://mlsanity.com/paper/2504.16722/) (2025)

## Abstract (arXiv)

Learning complex tasks from scratch is challenging and often impossible for humans as well as for artificial agents. A curriculum can be used instead, which decomposes a complex task (target task) into a sequence of source tasks (the curriculum). Each source task is a simplified version of the next source task with increasing complexity. Learning then occurs gradually by training on each source task while using knowledge from the curriculum's prior source tasks. In this study, we present a new algorithm that combines curriculum learning with Hindsight Experience Replay (HER), to learn sequential object manipulation tasks for multiple goals and sparse feedback. The algorithm exploits the recurrent structure inherent in many object manipulation tasks and implements the entire learning process in the original simulation without adjusting it to each source task. We have tested our algorithm on three challenging throwing tasks and show vast improvements compared to vanilla-HER.