# LiFT: How to Enable In-Context Learning for Longitudinal Modelling

arXiv: https://arxiv.org/abs/2604.16382 · HTML: https://mlsanity.com/paper/2604.16382/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 38 of its 63 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Builds on (most cited first, 30 of 38)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [Teaching Large Language Models to Reason with Reinforcement Learning](https://mlsanity.com/paper/2403.04642/) (2024)
- [Are Emergent Abilities in Large Language Models just In-Context Learning?](https://mlsanity.com/paper/2309.01809/) (2023)
- [Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning](https://mlsanity.com/paper/2406.09170/) (2024)
- [Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data](https://mlsanity.com/paper/2307.14385/) (2023)
- [All-in-one: Multi-task Learning for Rumour Verification](https://mlsanity.com/paper/1806.03713/) (2018)
- [Will Affective Computing Emerge from Foundation Models and General AI? A First Evaluation on ChatGPT](https://mlsanity.com/paper/2303.03186/) (2023)
- [TIMER: Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records](https://mlsanity.com/paper/2503.04176/) (2025)
- [Understanding In-Context Learning from Repetitions](https://mlsanity.com/paper/2310.00297/) (2023)
- [Are Large Language Models Temporally Grounded?](https://mlsanity.com/paper/2311.08398/) (2023)
- [TReMu: Towards Neuro-Symbolic Temporal Reasoning for LLM-Agents with Memory in Multi-Session Dialogues](https://mlsanity.com/paper/2502.01630/) (2025)
- [Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack](https://mlsanity.com/paper/2407.16695/) (2024)
- [Speaker Turn Modeling for Dialogue Act Classification](https://mlsanity.com/paper/2109.05056/) (2021)
- [ChronoSense: Exploring Temporal Understanding in Large Language Models with Time Intervals of Events](https://mlsanity.com/paper/2501.03040/) (2025)
- [PRIME: Large Language Model Personalization with Cognitive Dual-Memory and Personalized Thought Process](https://mlsanity.com/paper/2507.04607/) (2025)
- [Identifying Moments of Change from Longitudinal User Text](https://mlsanity.com/paper/2205.05593/) (2022)
- [Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Prediction](https://mlsanity.com/paper/2501.18724/) (2025)
- [Once Upon a textitTime in textitGraph: Relative-Time Pretraining for Complex Temporal Reasoning](https://mlsanity.com/paper/2310.14709/) (2023)
- [Combining Hierachical VAEs with LLMs for clinically meaningful timeline summarisation in social media](https://mlsanity.com/paper/2401.16240/) (2024)
- [Unlocking Temporal Question Answering for Large Language Models with Tailor-Made Reasoning Logic](https://mlsanity.com/paper/2305.15014/) (2023)
- [ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Data](https://mlsanity.com/paper/2505.14038/) (2025)
- [Towards Effective Time-Aware Language Representation: Exploring Enhanced Temporal Understanding in Language Models](https://mlsanity.com/paper/2406.01863/) (2024)

## Abstract (arXiv)

Longitudinal NLP tasks such as mental health monitoring and stance evolution require modeling temporally ordered text to track persistence and detect change. Such tasks also suffer from data scarcity, often involving rare events and sparsely annotated data. Large Language Models (LLMs) can learn from small amounts of data through in-context learning (ICL), which is particularly important in low-data resource scenarios. However, when it comes to tracking evolving interactions or identifying rare events or changes, LLMs fall short in both zero- and few-shot ICL. To address this limitation, we introduce LiFT, a model-agnostic longitudinal Instruction Fine-Tuning framework that is able to induce ICL behaviour in LLMs for longitudinal tasks. LiFT unifies diverse tasks through a shared instruction schema that leverages sequential dependencies, curriculum learning, and temporal conditioning. We evaluate LiFT on five longitudinal datasets under zero-, one-, and three-shot settings, demonstrating that LiFT delivers superior performance to IFT models under identical ICL conditions. Furthermore, we demonstrate that LiFT yields larger gains for minority classes, meaningfully attends to recent demonstration history, and develops transferable longitudinal modeling capabilities beyond its training tasks