In order to solve the problems that traditional visual SLAM networks have low accuracy and poor speed and lack semantic information, this paper proposed a novel RGB-D semantic segmentation network. The network used the depth information which less affected by the light in indoor scenes to improve the accuracy of segmentation, and meanwhile designed the lightweight multi-scale residual module(MRAM) and atrous spatial pyramid pooling(ASPP) module to lightweight the segmentation network and improve the segmentation accuracy. Firstly, the input image sequences entered the ORB-SLAM2 network to obtain keyframes. Then, the keyframes went into the proposed semantic segmentation network to get the 2D semantic label, and then it mapped the 2D semantic information to 3D pointcloud space. Finally, the method used the Bayesian algorithm to update the 3D map to obtain the globally consistent 3D pointcloud semantic map. The experiments adopted NYUv2 dataset to verify the performance of semantic segmentation network, and adopted TUM dataset construct pointcloud semantic map. The results show that the performance and speed of the semantic segmentation network in this paper are better than the existing models, and the combination of semantic segmentation network with visual SLAM can meet the requirements of constructing 3D dense semantic pointcloud map accurately and quickly.

[2022-06-18] Accepted Paper /doc/2022.04.0178/v1

[2022-11-05] Printed Article /doc/2022.04.0178/v2