Addressing the Sim2Real Gap in Robotic 3-D Object Classification

Publication Type:
Journal Article
IEEE Robotics and Automation Letters, 2020, 5, (2), pp. 407-413
Issue Date:
Filename Description Size
08931599.pdfPublished version1.93 MB
Adobe PDF
Full metadata record
© 2016 IEEE. Object classification with 3D data is an essential component of any scene understanding method. It has gained significant interest in a variety of communities, most notably in robotics and computer graphics. While the advent of deep learning has progressed the field of 3D object classification, most work using this data type are solely evaluated on CAD model datasets. Consequently, current work does not address the discrepancies existing between real and artificial data. In this work, we examine this gap in an indoor service robotic context by specifically addressing the problem of classification when transferring from artificial CAD models to real reconstructed objects. This is performed by training on ModelNet (CAD models) and evaluating on ScanNet (objects extracted from reconstructed rooms). We show that standard methods do not perform well in this task. We thus introduce a method that carefully samples object parts that are reproducible under various transformations and hence robust. Using graph convolution to classify the composed graph of parts, our method improves upon the baseline. Code is publicly available at
Please use this identifier to cite or link to this item: