Compared with RGB semantic segmentation, RGB-D semantic segmentation can combine the geometric depth information to effectively improve the segmentation accuracy. Considering the application of RGB-D semantic segmentation in autonomous driving, we design a real-time semantic segmentation network, that is, MAFFNet, which can effectively extract depth features and combine the complementary information in RGB and depth. We also design a multi-level attention feature fusion module that can excavate the available context information of RGB and depth features. At the same time, its inference speed can also meet the demands of autonomous driving. Experiments show that our network achieves excellent performance of 74.4% mIoU and an inference speed of 15.9 Hz at a full resolution of ${{2048}} \times {{1024}}$ on the cityscapes dataset. Using multi-source learning, we mixed the cityscapes and lost and found as the multi-dataset. Our network is also superior to previous algorithms in using the multi-dataset to detect small obstacles outside the road.
Data underlying the results presented in this article can be found in cityscapes [43] and lost and found [44].
43. M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 3213–3223.
44. P. Pinggera, S. Ramos, S. Gehrig, U. Franke, C. Rother, and R. Mester, “Lost and found: detecting small road hazards for self-driving vehicles,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2016), pp. 1099–1106.
Cited By
You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.