# UniHOI: Unified Human-Object Interaction Understanding via Unified Token Space

arXiv: https://arxiv.org/abs/2511.15046 · HTML: https://mlsanity.com/paper/2511.15046/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 34 references that are in the map.
- First posted on arXiv 19 Nov 2025.

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [Evaluating Text-to-Visual Generation with Image-to-Text Generation](https://mlsanity.com/paper/2404.01291/) (2024)
- [MMaDA: Multimodal Large Diffusion Language Models](https://mlsanity.com/paper/2505.15809/) (2025)
- [WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2503.07265/) (2025)
- [HOTR: End-to-End Human-Object Interaction Detection with Transformers](https://mlsanity.com/paper/2104.13682/) (2021)
- [Dense Text-to-Image Generation with Attention Modulation](https://mlsanity.com/paper/2308.12964/) (2023)
- [Visual Compositional Learning for Human-Object Interaction Detection](https://mlsanity.com/paper/2007.12407/) (2020)
- [FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition](https://mlsanity.com/paper/2312.07536/) (2023)
- [Open-World Human-Object Interaction Detection via Multi-modal Prompts](https://mlsanity.com/paper/2406.07221/) (2024)
- [LAION-SG: An Enhanced Large-Scale Dataset for Training Complex Image-Text Models with Structural Annotations](https://mlsanity.com/paper/2412.08580/) (2024)

## Built on by (most cited first, 10 of 10)

- [SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Models](https://mlsanity.com/paper/2601.03555/) (2026)
- [MUSE: Resolving Manifold Misalignment in Visual Tokenization via Topological Orthogonality](https://mlsanity.com/paper/2605.05646/) (2026)
- [GaitProtector: Impersonation-Driven Gait De-Identification via Training-Free Diffusion Latent Optimization](https://mlsanity.com/paper/2605.12431/) (2026)
- [SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendation](https://mlsanity.com/paper/2605.18920/) (2026)
- [Grounding Multi-Hop Reasoning in Structural Causal Models via Group Relative Policy Optimization](https://mlsanity.com/paper/2605.01482/) (2026)
- [Demystifying the Optimal Fair Classifier in Multi-Class Classification](https://mlsanity.com/paper/2606.00656/) (2026)
- [DiffuseAgent-MI: Distributionally-Grounded,Tool-Integrated Self-Evolving Agents for Faithful Visual Reasoning](https://mlsanity.com/paper/2608.00540/) (2026)
- [UniJEPA: A Unified Joint-Embedding Predictive Architecture for Task-Agnostic Visual World Modeling](https://mlsanity.com/paper/2608.07409/) (2026)
- [AWM-VLA: AlignedWorld Modeling for Efficient and Explainable Vision-Language-Action Policies](https://mlsanity.com/paper/2609.27753/) (2026)
- [HEIR: Learning Human-Entity Interactions with Functional Roles](https://mlsanity.com/paper/2609.35955/) (2026)

## Abstract (arXiv)

In the field of human-object interaction (HOI), detection and generation are two dual tasks that have traditionally been addressed separately, hindering the development of comprehensive interaction understanding. To address this, we propose UniHOI, which jointly models HOI detection and generation via a unified token space, thereby effectively promoting knowledge sharing and enhancing generalization. Specifically, we introduce a symmetric interaction-aware attention module and a unified semi-supervised learning paradigm, enabling effective bidirectional mapping between images and interaction semantics even under limited annotations. Extensive experiments demonstrate that UniHOI achieves state-of-the-art performance in both HOI detection and generation. Specifically, UniHOI improves accuracy by 4.9% on long-tailed HOI detection and boosts interaction metrics by 42.0% on open-vocabulary generation tasks.