# MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment

arXiv: https://arxiv.org/abs/2209.00244 · HTML: https://mlsanity.com/paper/2209.00244/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 64 references that are in the map.
- First posted on arXiv 1 Sep 2022.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding](https://mlsanity.com/paper/2203.00680/) (2022)
- [3D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detection](https://mlsanity.com/paper/2004.12636/) (2020)
- [A Deep Learning based No-reference Quality Assessment Model for UGC Videos](https://mlsanity.com/paper/2204.14047/) (2022)
- [Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection](https://mlsanity.com/paper/1903.01864/) (2019)
- [Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review](https://mlsanity.com/paper/2004.05224/) (2020)
- [Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspots](https://mlsanity.com/paper/1912.12791/) (2019)
- [Look, Listen, and Attend: Co-Attention Network for Self-Supervised Audio-Visual Representation Learning](https://mlsanity.com/paper/2008.05789/) (2020)
- [Inferring Point Cloud Quality via Graph Similarity](https://mlsanity.com/paper/2006.00497/) (2020)
- [PI-RCNN: An Efficient Multi-sensor 3D Object Detector with Point-based Attentive Cont-conv Fusion Module](https://mlsanity.com/paper/1911.06084/) (2019)
- [SSPC-Net: Semi-supervised Semantic 3D Point Cloud Segmentation Network](https://mlsanity.com/paper/2104.07861/) (2021)
- [Point Cloud Quality Assessment: Dataset Construction and Learning-based No-Reference Metric](https://mlsanity.com/paper/2012.11895/) (2020)
- [No-Reference Quality Assessment for 3D Colored Point Cloud and Mesh Models](https://mlsanity.com/paper/2107.02041/) (2021)
- [Perceptual Quality Assessment of Colored 3D Point Clouds](https://mlsanity.com/paper/2111.05474/) (2021)
- [No-Reference Point Cloud Quality Assessment via Domain Adaptation](https://mlsanity.com/paper/2112.02851/) (2021)
- [Reduced Reference Perceptual Quality Model and Application to Rate Control for 3D Point Cloud Compression](https://mlsanity.com/paper/2011.12688/) (2020)
- [Perceptual Quality Assessment of Omnidirectional Images](https://mlsanity.com/paper/2207.02674/) (2022)
- [Geometry Sharing Network for 3D Point Cloud Classification and Segmentation](https://mlsanity.com/paper/1912.10644/) (2019)
- [CAT-Det: Contrastively Augmented Transformer for Multi-modal 3D Object Detection](https://mlsanity.com/paper/2204.00325/) (2022)

## Built on by (most cited first, 30 of 39)

- [Perceptual Video Quality Assessment: A Survey](https://mlsanity.com/paper/2402.03413/) (2024)
- [Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluation](https://mlsanity.com/paper/2307.02808/) (2023)
- [Quality Assessment in the Era of Large Models: A Survey](https://mlsanity.com/paper/2409.00031/) (2024)
- [Subjective and Objective Quality Assessment for in-the-Wild Computer Graphics Images](https://mlsanity.com/paper/2303.08050/) (2023)
- [GMS-3DQA: Projection-based Grid Mini-patch Sampling for 3D Model Quality Assessment](https://mlsanity.com/paper/2306.05658/) (2023)
- [LMM-PCQA: Assisting Point Cloud Quality Assessment with LMM](https://mlsanity.com/paper/2404.18203/) (2024)
- [Contrastive Pre-Training with Multi-View Fusion for No-Reference Point Cloud Quality Assessment](https://mlsanity.com/paper/2403.10066/) (2024)
- [Perceptual Quality Assessment for Digital Human Heads](https://mlsanity.com/paper/2209.09489/) (2022)
- [DDH-QA: A Dynamic Digital Humans Quality Assessment Database](https://mlsanity.com/paper/2212.12734/) (2022)
- [EEP-3DQA: Efficient and Effective Projection-based 3D Model Quality Assessment](https://mlsanity.com/paper/2302.08715/) (2023)
- [Dual-Branch Network for Portrait Image Quality Assessment](https://mlsanity.com/paper/2405.08555/) (2024)
- [Geometry-Aware Video Quality Assessment for Dynamic Digital Human](https://mlsanity.com/paper/2310.15984/) (2023)
- [CLIP-PCQA: Exploring Subjective-Aligned Vision-Language Modeling for Point Cloud Quality Assessment](https://mlsanity.com/paper/2501.10071/) (2025)
- [A No-Reference Quality Assessment Method for Digital Human Head](https://mlsanity.com/paper/2310.16732/) (2023)
- [GeodesicPSIM: Predicting the Quality of Static Mesh with Texture Map via Geodesic Patch Similarity](https://mlsanity.com/paper/2308.04928/) (2023)
- [PointPCA+: Extending PointPCA objective quality assessment metric](https://mlsanity.com/paper/2311.13880/) (2023)
- [Stochasticity-aware No-Reference Point Cloud Quality Assessment](https://mlsanity.com/paper/2401.08926/) (2024)
- [PAME: Self-Supervised Masked Autoencoder for No-Reference Point Cloud Quality Assessment](https://mlsanity.com/paper/2403.10061/) (2024)
- [Image-Guided Outdoor LiDAR Perception Quality Assessment for Autonomous Driving](https://mlsanity.com/paper/2406.17265/) (2024)
- [PIT-QMM: A Large Multimodal Model For No-Reference Point Cloud Quality Assessment](https://mlsanity.com/paper/2510.07636/) (2025)
- [Asynchronous Feedback Network for Perceptual Point Cloud Quality Assessment](https://mlsanity.com/paper/2407.09806/) (2024)
- [Learning Disentangled Representations for Perceptual Point Cloud Quality Assessment via Mutual Information Minimization](https://mlsanity.com/paper/2411.07936/) (2024)
- [HybridMQA: Exploring Geometry-Texture Interactions for Colored Mesh Quality Assessment](https://mlsanity.com/paper/2412.01986/) (2024)
- [From Images to Point Clouds: An Efficient Solution for Cross-media Blind Quality Assessment without Annotated Training](https://mlsanity.com/paper/2501.13387/) (2025)
- [MeshFleet: Filtered and Annotated 3D Vehicle Dataset for Domain Specific Generative Modeling](https://mlsanity.com/paper/2503.14002/) (2025)
- [Low-Complexity Patch-based No-Reference Point Cloud Quality Metric exploiting Weighted Structure and Texture Features](https://mlsanity.com/paper/2503.15001/) (2025)
- [Point Cloud Compression and Objective Quality Assessment: A Survey](https://mlsanity.com/paper/2506.22902/) (2025)
- [UPDA: Unsupervised Progressive Domain Adaptation for No-Reference Point Cloud Quality Assessment](https://mlsanity.com/paper/2602.11969/) (2026)
- [Activating Frequency and ViT for 3D Point Cloud Quality Assessment without Reference](https://mlsanity.com/paper/2312.05972/) (2023)
- [Perceptual Quality Assessment of Trisoup-Lifting Encoded 3D Point Clouds](https://mlsanity.com/paper/2410.06689/) (2024)

## Abstract (arXiv)

The visual quality of point clouds has been greatly emphasized since the ever-increasing 3D vision applications are expected to provide cost-effective and high-quality experiences for users. Looking back on the development of point cloud quality assessment (PCQA) methods, the visual quality is usually evaluated by utilizing single-modal information, i.e., either extracted from the 2D projections or 3D point cloud. The 2D projections contain rich texture and semantic information but are highly dependent on viewpoints, while the 3D point clouds are more sensitive to geometry distortions and invariant to viewpoints. Therefore, to leverage the advantages of both point cloud and projected image modalities, we propose a novel no-reference point cloud quality assessment (NR-PCQA) metric in a multi-modal fashion. In specific, we split the point clouds into sub-models to represent local geometry distortions such as point shift and down-sampling. Then we render the point clouds into 2D image projections for texture feature extraction. To achieve the goals, the sub-models and projected images are encoded with point-based and image-based neural networks. Finally, symmetric cross-modal attention is employed to fuse multi-modal quality-aware information. Experimental results show that our approach outperforms all compared state-of-the-art methods and is far ahead of previous NR-PCQA methods, which highlights the effectiveness of the proposed method. The code is available at https://github.com/zzc-1998/MM-PCQA.