# FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstruction

arXiv: https://arxiv.org/abs/2206.02194 · HTML: https://mlsanity.com/paper/2206.02194/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 5 Jun 2022.

## Builds on (most cited first, 26 of 26)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [ICON: Implicit Clothed humans Obtained from Normals](https://mlsanity.com/paper/2112.09127/) (2021)
- [PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop](https://mlsanity.com/paper/2103.16507/) (2021)
- [Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors](https://mlsanity.com/paper/2105.01859/) (2021)
- [PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstruction](https://mlsanity.com/paper/2007.03858/) (2020)
- [STAR: Sparse Trained Articulated Human Body Regressor](https://mlsanity.com/paper/2008.08535/) (2020)
- [DeepHuman: 3D Human Reconstruction from a Single Image](https://mlsanity.com/paper/1903.06473/) (2019)
- [ARCH++: Animation-Ready Clothed Human Reconstruction Revisited](https://mlsanity.com/paper/2108.07845/) (2021)
- [Tex2Shape: Detailed Full Human Body Geometry From a Single Image](https://mlsanity.com/paper/1904.08645/) (2019)
- [DeepCap: Monocular Human Performance Capture Using Weak Supervision](https://mlsanity.com/paper/2003.08325/) (2020)
- [Collaborative Regression of Expressive Bodies using Moderation](https://mlsanity.com/paper/2105.05301/) (2021)
- [Robust High-Resolution Video Matting with Temporal Guidance](https://mlsanity.com/paper/2108.11515/) (2021)
- [Photorealistic Monocular 3D Reconstruction of Humans Wearing Clothing](https://mlsanity.com/paper/2204.08906/) (2022)
- [Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction](https://mlsanity.com/paper/2006.08072/) (2020)
- [Recovering 3D Human Mesh from Monocular Images: A Survey](https://mlsanity.com/paper/2203.01923/) (2022)
- [Monocular Real-Time Volumetric Performance Capture](https://mlsanity.com/paper/2007.13988/) (2020)
- [Detailed Human Shape Estimation from a Single Image by Hierarchical Mesh Deformation](https://mlsanity.com/paper/1904.10506/) (2019)
- [Detailed Avatar Recovery from Single Image](https://mlsanity.com/paper/2108.02931/) (2021)

## Built on by (most cited first, 27 of 27)

- [Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition](https://mlsanity.com/paper/2302.11566/) (2023)
- [SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion](https://mlsanity.com/paper/2311.15855/) (2023)
- [IDOL: Instant Photorealistic 3D Human Creation from a Single Image](https://mlsanity.com/paper/2412.14963/) (2024)
- [Representing Volumetric Videos as Dynamic MLP Maps](https://mlsanity.com/paper/2304.06717/) (2023)
- [Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models](https://mlsanity.com/paper/2406.08475/) (2024)
- [Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Prior](https://mlsanity.com/paper/2503.01610/) (2025)
- [HumanRef: Single Image to 3D Human Generation via Reference-Guided Diffusion](https://mlsanity.com/paper/2311.16961/) (2023)
- [Gen-3Diffusion: Realistic Image-to-3D Generation via 2D & 3D Diffusion Synergy](https://mlsanity.com/paper/2412.06698/) (2024)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)
- [Learning Visibility Field for Detailed 3D Human Reconstruction and Relighting](https://mlsanity.com/paper/2304.11900/) (2023)
- [FourierHandFlow: Neural 4D Hand Representation Using Fourier Query Flow](https://mlsanity.com/paper/2307.08100/) (2023)
- [Real-time Free-view Human Rendering from Sparse-view RGB Videos using Double Unprojected Textures](https://mlsanity.com/paper/2412.13183/) (2024)
- [Joint2Human: High-quality 3D Human Generation via Compact Spherical Embedding of 3D Joints](https://mlsanity.com/paper/2312.08591/) (2023)
- [High-Quality Animatable Dynamic Garment Reconstruction from Monocular Videos](https://mlsanity.com/paper/2311.01214/) (2023)
- [R2Human: Real-Time 3D Human Appearance Rendering from a Single Image](https://mlsanity.com/paper/2312.05826/) (2023)
- [Occupancy Planes for Single-view RGB-D Human Reconstruction](https://mlsanity.com/paper/2208.02817/) (2022)
- [Occupancy-Based Dual Contouring](https://mlsanity.com/paper/2409.13418/) (2024)
- [SMPL Normal Map Is All You Need for Single-view Textured Human Reconstruction](https://mlsanity.com/paper/2506.12793/) (2025)
- [An Embeddable Implicit IUVD Representation for Part-based 3D Human Surface Reconstruction](https://mlsanity.com/paper/2401.16810/) (2024)
- [MultiGO++: Monocular 3D Clothed Human Reconstruction via Geometry-Texture Collaboration](https://mlsanity.com/paper/2603.04993/) (2026)
- [Weakly-Supervised 3D Reconstruction of Clothed Humans via Normal Maps](https://mlsanity.com/paper/2311.16042/) (2023)
- [AdaptiveFusion: Adaptive Multi-Modal Multi-View Fusion for 3D Human Body Reconstruction](https://mlsanity.com/paper/2409.04851/) (2024)
- [FOF-X: Towards Real-time Detailed Human Reconstruction from a Single Image](https://mlsanity.com/paper/2412.05961/) (2024)
- [Unify3D: An Augmented Holistic End-to-end Monocular 3D Human Reconstruction via Anatomy Shaping and Twins Negotiating](https://mlsanity.com/paper/2504.18215/) (2025)
- [Disambiguating Monocular Reconstruction of 3D Clothed Human with Spatial-Temporal Transformer](https://mlsanity.com/paper/2410.16337/) (2024)
- [Link to the Past: Temporal Propagation for Fast 3D Human Reconstruction from Monocular Video](https://mlsanity.com/paper/2505.07333/) (2025)
- [OAHuman: Occlusion-Aware 3D Human Reconstruction from Monocular Images](https://mlsanity.com/paper/2603.14249/) (2026)

## Abstract (arXiv)

The advent of deep learning has led to significant progress in monocular human reconstruction. However, existing representations, such as parametric models, voxel grids, meshes and implicit neural representations, have difficulties achieving high-quality results and real-time speed at the same time. In this paper, we propose Fourier Occupancy Field (FOF), a novel powerful, efficient and flexible 3D representation, for monocular real-time and accurate human reconstruction. The FOF represents a 3D object with a 2D field orthogonal to the view direction where at each 2D position the occupancy field of the object along the view direction is compactly represented with the first few terms of Fourier series, which retains the topology and neighborhood relation in the 2D domain. A FOF can be stored as a multi-channel image, which is compatible with 2D convolutional neural networks and can bridge the gap between 3D geometries and 2D images. The FOF is very flexible and extensible, e.g., parametric models can be easily integrated into a FOF as a prior to generate more robust results. Based on FOF, we design the first 30+FPS high-fidelity real-time monocular human reconstruction framework. We demonstrate the potential of FOF on both public dataset and real captured data. The code will be released for research purposes.