Surface-type classification in structured planar environments under various illumination and imaging conditions
- Publication Type:
- Issue Date:
The recent advancement in sensing, computing and artificial intelligence, has led to the application of robots outside of the manufacturing factory and into field environments. In order for a field robot to operate intelligently and autonomously, the robot needs to build an environmental awareness, such as by classifying the different surface-types on a steel bridge structure. However, it is challenging to classify surface-types from images that are captured in a structurally complex environment under various illumination and imaging conditions. This is because colour and texture features extracted from these images can be inconsistent. This thesis presents a surface-type classification approach to classify surface-types in a structurally complex three-dimensional (3D) environment under various illumination and imaging conditions. The approach proposes RGB-D sensing to provide each pixel in an image with additional depth information that is used by two developed algorithms. The first algorithm uses the RGB-D information along with a modified reflectance model to extract colour features for colour-based classification of surface-types. The second algorithm uses the depth information to calculate a probability map for the pixels being a specific surface-type. The probability map can identify the image regions that have a high probability of being accurately classified by a texture-based classifier. A 3D grid-based map is generated to combine the results produced by colour-based classification and texture-based classification. It is suggested that a robot manipulator is used to position an RGB-D sensor package in the complex environments to capture the RGB-D images. In this way, the 3D position of each pixel is precisely known in a common global frame (robot base coordinate frame) and can be combined using a grid-based map to build up a rich awareness of the surrounding complex environment. A case study is conducted in a laboratory environment using a six degree-of-freedom robot manipulator equipped with a RGB-D sensor package mounted to the end effector. The results show that the proposed surface-type classification approach provides an improved solution for vision-based classification of surface-types in a complex structural environment with various illumination and imaging conditions.
Please use this identifier to cite or link to this item: