# HilDA: Hierarchical Distillation with Diffusion for Advancing Self-Supervised LiDAR Pre-training

arXiv: https://arxiv.org/abs/2606.20189 · HTML: https://mlsanity.com/paper/2606.20189/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 107 references that are in the map.
- First posted on arXiv 18 Jun 2026.

## Builds on (most cited first, 5 of 5)

- [Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail](https://mlsanity.com/paper/2511.00088/) (2025)
- [Auxiliary Tasks in Multi-task Learning](https://mlsanity.com/paper/1805.06334/) (2018)
- [DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model](https://mlsanity.com/paper/2410.10429/) (2024)
- [Learning from 2D: Contrastive Pixel-to-Point Knowledge Transfer for 3D Pretraining](https://mlsanity.com/paper/2104.04687/) (2021)
- [BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latents](https://mlsanity.com/paper/2407.05679/) (2024)

## Abstract (arXiv)

Leveraging Vision Foundation Models (VFMs) for camera-to-LiDAR knowledge distillation offers a promising solution to the scarcity of annotated data needed to represent the immense geometric and kinematic diversity of real-world autonomous driving (AD). However, current approaches typically treat VFMs as black-box teachers, relying exclusively on frame-wise feature similarity. Consequently, they do not fully exploit the teacher's layer-wise semantic structure and global context, as well as the rich spatiotemporal information inherent in LiDAR sequences. We propose HilDA, a self-supervised pretraining framework for LiDAR backbones that better captures the semantic what and geometric where needed for driving tasks. HilDA combines hierarchical distillation comprising multi-layer distillation for progressive semantic alignment and global context distillation for scene-level semantics, with a temporal occupancy diffusion objective promoting spatiotemporal consistency. Models pre-trained with HilDA achieve state-of-the-art results on cross-modal distillation benchmarks and outperform models trained via prior distillation approaches on 3D object detection, scene flow, and semantic occupancy prediction. Code available at: https://maxiuw.github.io/hilda.