Query-guided Support Prototypes for Few-shot 3D Indoor Segmentation

Abstract

Few-shot 3D point cloud segmentation segments novel categories in point cloud scenes with only limited annotations. However, most current methods do not consider query content when exploring support prototypes, and thus suffer from intra-class variations between objects and incomplete representation of category information from annotated support samples. In this paper, we propose a novel Query-Guided support Prototype exploration Network (QGPNet) to tackle this challenge. Firstly, we present a point feature alignment module, which leverages geometry relationship between prototypes and query points, to tackle data misalignment caused by intra-class variations, and thus prevents incorrect label propagation from prototypes to query points. Secondly, we design a prototype feature mining strategy, which progressively harvests diverse support prototypes in the interaction with query features, to fully utilize the category information provided by annotated samples. Additionally, we introduce a semantic-aware data augmentation strategy for query samples in the training process, potentially improving the generalization ability of support prototypes on query samples. Extensive experiments on two indoor 3D datasets S3DIS and ScanNet demonstrate that QGPNet outperforms previous state-of-the-art methods by a large margin.