Abstract

The cutting-edge imaging system exhibits low output resolution and high power consumption, presenting challenges for the RGB-D fusion algorithm. In practical scenarios, aligning the depth map resolution with the RGB image sensor is a crucial requirement. In this Letter, the software and hardware co-design is considered to implement a lidar system based on the monocular RGB 3D imaging algorithm. A 6.4 × 6.4-mm² deep-learning accelerator (DLA) system-on-chip (SoC) manufactured in a 40-nm CMOS is incorporated with a 3.6-mm² TX-RX integrated chip fabricated in a 180-nm CMOS to employ the customized single-pixel imaging neural network. In comparison to the RGB-only monocular depth estimation technique, the root mean square error is reduced from 0.48 m to 0.3 m on the evaluated dataset, and the output depth map resolution matches the RGB input.

© 2023 Optica Publishing Group

80 × 120 AI-enhanced LiDAR system based on a lightweight intensity–RGB–dToF sensor fusion neural network deployed on an edge device

Lebei Cui, Jie Li, Shenglong Zhuo, Yifan Wu, Sifan Zhou, Jian Qian, Miao Sun, Jier Wang, Patrick Yin Chiang, and Yun Chen
Opt. Lett. 48(23) 6192-6195 (2023)

Monocular depth estimation network with single-pixel depth guidance

Hongjae Lee, Jinbum Park, Wooseok Jeong, and Seung-Won Jung
Opt. Lett. 48(3) 594-597 (2023)

Super-resolution multimode fiber imaging with an untrained neural network

Wei Li, Ksenia Abrashitova, and Lyubov V. Amitonova
Opt. Lett. 48(13) 3363-3366 (2023)

SPADnet: deep RGB-SPAD sensor fusion assisted by monocular depth estimation

Zhanghao Sun, David B. Lindell, Olav Solgaard, and Gordon Wetzstein
Opt. Express 28(10) 14948-14962 (2020)

Method of depth simulation imaging and depth image super-resolution reconstruction for a 2D/3D compatible CMOS image sensor

Shijie Guo, Quanmin Chen, Zhe Zhao, Jiangtao Xu, and Kaiming Nie
Appl. Opt. 62(17) 4439-4454 (2023)