# F-CAD: A Framework to Explore Hardware Accelerators for Codec Avatar Decoding

arXiv: https://arxiv.org/abs/2103.04958 · HTML: https://mlsanity.com/paper/2103.04958/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 19 references that are in the map.
- First posted on arXiv 8 Mar 2021.

## Builds on (most cited first, 8 of 8)

- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Deep Appearance Models for Face Rendering](https://mlsanity.com/paper/1808.00362/) (2018)
- [Audio- and Gaze-driven Facial Animation of Codec Avatars](https://mlsanity.com/paper/2008.05023/) (2020)
- [AutoDNNchip: An Automated DNN Chip Predictor and Builder for Both FPGAs and ASICs](https://mlsanity.com/paper/2001.03535/) (2020)
- [DNNExplorer: A Framework for Modeling and Exploring a Novel Paradigm of FPGA-based DNN Accelerator](https://mlsanity.com/paper/2008.12745/) (2020)
- [Expressive Telepresence via Modular Codec Avatars](https://mlsanity.com/paper/2008.11789/) (2020)
- [HybridDNN: A Framework for High-Performance Hybrid DNN Accelerator Design and Implementation](https://mlsanity.com/paper/2004.03804/) (2020)
- [OO-VR: NUMA Friendly Object-Oriented VR Rendering Framework For Future NUMA-Based Multi-GPU Systems](https://mlsanity.com/paper/2001.03537/) (2020)

## Built on by (most cited first, 2 of 2)

- [Unlocking Visual Secrets: Inverting Features with Diffusion Priors for Image Reconstruction](https://mlsanity.com/paper/2412.10448/) (2024)
- [You Only Search Once: On Lightweight Differentiable Architecture Search for Resource-Constrained Embedded Platforms](https://mlsanity.com/paper/2208.14446/) (2022)

## Abstract (arXiv)

Creating virtual avatars with realistic rendering is one of the most essential and challenging tasks to provide highly immersive virtual reality (VR) experiences. It requires not only sophisticated deep neural network (DNN) based codec avatar decoders to ensure high visual quality and precise motion expression, but also efficient hardware accelerators to guarantee smooth real-time rendering using lightweight edge devices, like untethered VR headsets. Existing hardware accelerators, however, fail to deliver sufficient performance and efficiency targeting such decoders which consist of multi-branch DNNs and require demanding compute and memory resources. To address these problems, we propose an automation framework, called F-CAD (Facebook Codec avatar Accelerator Design), to explore and deliver optimized hardware accelerators for codec avatar decoding. Novel technologies include 1) a new accelerator architecture to efficiently handle multi-branch DNNs; 2) a multi-branch dynamic design space to enable fine-grained architecture configurations; and 3) an efficient architecture search for picking the optimized hardware design based on both application-specific demands and hardware resource constraints. To the best of our knowledge, F-CAD is the first automation tool that supports the whole design flow of hardware acceleration of codec avatar decoders, allowing joint optimization on decoder designs in popular machine learning frameworks and corresponding customized accelerator design with cycle-accurate evaluation. Results show that the accelerators generated by F-CAD can deliver up to 122.1 frames per second (FPS) and 91.6% hardware efficiency when running the latest codec avatar decoder. Compared to the state-of-the-art designs, F-CAD achieves 4.0X and 2.8X higher throughput, 62.5% and 21.2% higher efficiency than DNNBuilder and HybridDNN by targeting the same hardware device.