# Short-Term-to-Long-Term Memory Transfer for Knowledge Graphs under Partial Observability

arXiv: https://arxiv.org/abs/2605.22142 · HTML: https://mlsanity.com/paper/2605.22142/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 21 May 2026.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Neural Episodic Control](https://mlsanity.com/paper/1703.01988/) (2017)
- [Message Passing for Hyper-Relational Knowledge Graphs](https://mlsanity.com/paper/2009.10847/) (2020)
- [Deep Transformer Q-Networks for Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2206.01078/) (2022)
- [On overfitting and asymptotic bias in batch reinforcement learning with partial observability](https://mlsanity.com/paper/1709.07796/) (2017)
- [Towards Symbolic Reinforcement Learning with Common Sense](https://mlsanity.com/paper/1804.08597/) (2018)
- [The act of remembering: a study in partially observable reinforcement learning](https://mlsanity.com/paper/2010.01753/) (2020)
- [A Machine with Short-Term, Episodic, and Semantic Memory Systems](https://mlsanity.com/paper/2212.02098/) (2022)
- [Partially Observable Reinforcement Learning with Memory Traces](https://mlsanity.com/paper/2503.15200/) (2025)
- [Temporal Knowledge-Graph Memory in a Partially Observable Environment](https://mlsanity.com/paper/2408.05861/) (2024)
- [Neuro-symbolic Action Masking for Deep Reinforcement Learning](https://mlsanity.com/paper/2602.10598/) (2026)

## Abstract (arXiv)

Reinforcement learning under partial observability requires deciding what information to retain, yet most memory-based approaches do not explicitly model short-term-to-long-term transfer of symbolic observations. We study this transfer process in a temporal knowledge-graph memory setting and cast it as a neuro-symbolic value-based decision problem: for each observed triple, the agent chooses whether to keep or drop it before long-term insertion. To handle variable-sized short-term buffers, we use a per-item Q-learning design with shared parameters and a practical temporal-difference update over matched items across consecutive steps. On the RoomKG benchmark at long-term memory capacity 128, learned transfer decisions outperform symbolic and neural baselines, including symbolic baselines with temporal annotations and history-based LSTM/Transformer baselines. Across transfer-policy ablations, a lightweight local short-term-only variant performs best, and step-level behavior shows that the policy keeps navigation- and query-relevant facts while discarding lower-value candidate facts, supporting explicit and interpretable memory decisions under memory constraints.