# LMM-PCQA: Assisting Point Cloud Quality Assessment with LMM

arXiv: https://arxiv.org/abs/2404.18203 · HTML: https://mlsanity.com/paper/2404.18203/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 62 references that are in the map.
- First posted on arXiv 28 Apr 2024.

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality](https://mlsanity.com/paper/2304.14178/) (2023)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration](https://mlsanity.com/paper/2311.04257/) (2023)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision](https://mlsanity.com/paper/2309.14181/) (2023)
- [Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models](https://mlsanity.com/paper/2311.06783/) (2023)
- [Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models](https://mlsanity.com/paper/2312.08962/) (2023)
- [Towards Open-ended Visual Quality Comparison](https://mlsanity.com/paper/2402.16641/) (2024)
- [Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspots](https://mlsanity.com/paper/1912.12791/) (2019)
- [Inferring Point Cloud Quality via Graph Similarity](https://mlsanity.com/paper/2006.00497/) (2020)
- [PI-RCNN: An Efficient Multi-sensor 3D Object Detector with Point-based Attentive Cont-conv Fusion Module](https://mlsanity.com/paper/1911.06084/) (2019)
- [SSPC-Net: Semi-supervised Semantic 3D Point Cloud Segmentation Network](https://mlsanity.com/paper/2104.07861/) (2021)
- [Point Cloud Quality Assessment: Dataset Construction and Learning-based No-Reference Metric](https://mlsanity.com/paper/2012.11895/) (2020)
- [No-Reference Quality Assessment for 3D Colored Point Cloud and Mesh Models](https://mlsanity.com/paper/2107.02041/) (2021)
- [MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment](https://mlsanity.com/paper/2209.00244/) (2022)
- [No-Reference Point Cloud Quality Assessment via Domain Adaptation](https://mlsanity.com/paper/2112.02851/) (2021)
- [A No-reference Quality Assessment Metric for Point Cloud Based on Captured Video Sequences](https://mlsanity.com/paper/2206.05054/) (2022)
- [Learning a No-Reference Quality Assessment Model of Enhanced Images With Big Data](https://mlsanity.com/paper/1904.08632/) (2019)
- [D-DPCC: Deep Dynamic Point Cloud Compression via 3D Motion Prediction](https://mlsanity.com/paper/2205.01135/) (2022)
- [VisualCritic: Making LMMs Perceive Visual Quality Like Humans](https://mlsanity.com/paper/2403.12806/) (2024)
- [PointPCA: Point Cloud Objective Quality Assessment Using PCA-Based Descriptors](https://mlsanity.com/paper/2111.12663/) (2021)
- [PointPCA+: Extending PointPCA objective quality assessment metric](https://mlsanity.com/paper/2311.13880/) (2023)

## Built on by (most cited first, 19 of 19)

- [Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMs](https://mlsanity.com/paper/2409.20063/) (2024)
- [Quality Assessment in the Era of Large Models: A Survey](https://mlsanity.com/paper/2409.00031/) (2024)
- [RadarQA: Multi-modal Quality Analysis of Weather Radar Forecasts](https://mlsanity.com/paper/2508.12291/) (2025)
- [CMC-Bench: Towards a New Paradigm of Visual Signal Compression](https://mlsanity.com/paper/2406.09356/) (2024)
- [3DGCQA: A Quality Assessment Database for 3D AI-Generated Contents](https://mlsanity.com/paper/2409.07236/) (2024)
- [AGAV-Rater: Adapting Large Multimodal Model for AI-Generated Audio-Visual Quality Assessment](https://mlsanity.com/paper/2501.18314/) (2025)
- [Engagement Prediction of Short Videos with Large Multimodal Models](https://mlsanity.com/paper/2508.02516/) (2025)
- [Q-Save: Towards Scoring and Attribution for Generated Video Evaluation](https://mlsanity.com/paper/2511.18825/) (2025)
- [DHQA-4D: Perceptual Quality Assessment of Dynamic 4D Digital Human](https://mlsanity.com/paper/2510.03874/) (2025)
- [PIT-QMM: A Large Multimodal Model For No-Reference Point Cloud Quality Assessment](https://mlsanity.com/paper/2510.07636/) (2025)
- [HybridMQA: Exploring Geometry-Texture Interactions for Colored Mesh Quality Assessment](https://mlsanity.com/paper/2412.01986/) (2024)
- [Point Cloud Compression and Objective Quality Assessment: A Survey](https://mlsanity.com/paper/2506.22902/) (2025)
- [OmniQuality-R: Advancing Reward Models Through All-Encompassing Quality Assessment](https://mlsanity.com/paper/2510.10609/) (2025)
- [UPDA: Unsupervised Progressive Domain Adaptation for No-Reference Point Cloud Quality Assessment](https://mlsanity.com/paper/2602.11969/) (2026)
- [Perceptual Quality Assessment of Trisoup-Lifting Encoded 3D Point Clouds](https://mlsanity.com/paper/2410.06689/) (2024)
- [PointQ-Bench: Benchmarking Diagnostic and Interpretable Point Cloud Quality Assessment](https://mlsanity.com/paper/2605.28241/) (2026)
- [DAL-PCQA: Enabling Distortion-Level and Language-Driven Reasoning for Point Cloud Quality Assessment](https://mlsanity.com/paper/2606.07938/) (2026)
- [PCQA-R1: Advancing Generalized 3D Point Cloud Quality Assessment with Reinforcement Learning](https://mlsanity.com/paper/2608.18627/) (2026)
- [LLaVA-Assessor: Building the Foundation LMM For Visual Quality Assessment](https://mlsanity.com/paper/2609.26205/) (2026)

## Abstract (arXiv)

Although large multi-modality models (LMMs) have seen extensive exploration and application in various quality assessment studies, their integration into Point Cloud Quality Assessment (PCQA) remains unexplored. Given LMMs' exceptional performance and robustness in low-level vision and quality assessment tasks, this study aims to investigate the feasibility of imparting PCQA knowledge to LMMs through text supervision. To achieve this, we transform quality labels into textual descriptions during the fine-tuning phase, enabling LMMs to derive quality rating logits from 2D projections of point clouds. To compensate for the loss of perception in the 3D domain, structural features are extracted as well. These quality logits and structural features are then combined and regressed into quality scores. Our experimental results affirm the effectiveness of our approach, showcasing a novel integration of LMMs into PCQA that enhances model understanding and assessment accuracy. We hope our contributions can inspire subsequent investigations into the fusion of LMMs with PCQA, fostering advancements in 3D visual quality analysis and beyond. The code is available at https://github.com/zzc-1998/LMM-PCQA.