click here to download the project abstract/base paper of 3d shapes Final year projects
ABSTRACT
As two fundamental representation modalities of 3d shapes final year project in Advanced Deep Learning of 3D objects, 3D point clouds and multi-view 2D images record shape information from different domains of geometric structures and visual appearances. In the current deep learning era, remarkable progress in processing such two data modalities has been achieved through customizing compatible 3D and 2D network architectures. However, unlike multi-view image-based 2D visual modeling paradigms, which have shown leading performance in several common 3D shape recognition benchmarks, point cloud-based 3D geometric modeling paradigms are still highly limited by insufficient learning capacity, due to the difficulty of extracting discriminative features from irregular geometric signals. This paper explores the possibility of boosting deep 3D point cloud encoders by transferring visual knowledge extracted from deep 2D image encoders under a standard teacher-student distillation workflow. Generally, we propose PointMCD, a unified multi-view cross-modal distillation architecture, including a pre-trained deep image encoder as the teacher and a deep point encoder as the student. To perform heterogeneous feature alignment between 2D visual and 3D geometric domains, we further investigate visibility-aware feature projection (VAFP), by which point-wise embeddings are reasonably aggregated into view-specific geometric descriptors. By pair-wisely aligning multi-view visual and geometric descriptors, we can obtain more powerful deep point encoders without exhausting and complicated network modification. Experiments on 3D shape classification, part segmentation, and unsupervised learning strongly validate the effectiveness of our method. The code and data will be publicly available at https://github.com/keeganhk/PointMCD.ield of computer science and artificial intelligence will find POINTMCD to be an insightful exploration of cutting-edge techniques in deep learning. The paper presents a novel methodology and contributes to the broader landscape of 3D shape recognition, opening avenues for further research and development. Engage with POINTMCD to deepen your understanding of state-of-the-art approaches in the exciting realm of point cloud analysis.
Indexed-Terms: 3D point cloud, multi-view images, cross-modal, knowledge distillation, 3D shape recognition
