Abstract

Point clouds have been widely used due to their information being richer than images. Fringe projection profilometry (FPP) is one of the camera-based point cloud acquisition techniques that is being developed as a vision system for robotic surgery. For semi-autonomous robotic suturing, fluorescent fiducials were previously used on a target tissue as suture landmarks. This not only increases system complexity but also imposes safety concerns. To address these problems, we propose a numerical landmark localization algorithm based on a convolutional neural network (CNN) and a conditional random field (CRF). A CNN is applied to regress landmark heatmaps from the four-channel image data generated by the FPP. A CRF leveraging both local and global shape constraints is developed to better tune the landmark coordinates, reject extra landmarks, and recover missing landmarks. The robustness of the proposed method is demonstrated through ex vivo porcine intestine landmark localization experiments.

© 2022 Optica Publishing Group

FPP-SLAM: indoor simultaneous localization and mapping based on fringe projection profilometry

Yang Zhao, Haotian Yu, Kai Zhang, Yucheng Zheng, Yi Zhang, Dongliang Zheng, and Jing Han
Opt. Express 31(4) 5853-5871 (2023)

Fringe projection profilometry by conducting deep learning from its digital twin

Yi Zheng, Shaodong Wang, Qing Li, and Beiwen Li
Opt. Express 28(24) 36568-36583 (2020)

Single-shot fringe projection profilometry based on deep learning and computer graphics

Fanzhou Wang, Chenxing Wang, and Qingze Guan
Opt. Express 29(6) 8024-8040 (2021)

Deep learning-enabled anti-ambient light approach for fringe projection profilometry

Ruihu Zhang, Minghui Duan, Xin Fan, Yabing Zheng, Zheng Sun, Jinjin Zheng, and Yi Jin
Opt. Express 30(26) 47672-47689 (2022)

Accurate calibration for fringe projection profilometry based on an improved subpixel mapping with local gray distribution

Bin Liu, Xinxin He, Chunliu Wang, Sen Wang, and Guanhao Wu
J. Opt. Soc. Am. A 40(12) 2164-2176 (2023)