Open Access
Add to BibSonomy
Abstract
Single-photon light detection and ranging (LiDAR) is widely used to reconstruct 3D scenes. Nevertheless, depth and reflectivity maps obtained by single-photon detection usually suffer from noise problems. Threshold LiDAR techniques using photon-number-resolving detectors were proposed to suppress noise by filtering low photon numbers, but these techniques renounce multiple levels of information and could not be compatible when it comes to high-noise low-signal regime. In this manuscript, we propose a detection scheme which combines the noise suppression of threshold detection with the signal amplification of photon-number-resolving detectors to further enhance LiDAR performance. The enhancement attained is compared to single-photon and threshold detection schemes under a wide range of signal and noise conditions, in terms of signal-to-noise-ratio (SNR), detection rate and false alarm rate, which are key metrics for LiDAR. Extensive simulations and real-world experiments show that the proposed scheme can reconstruct better depth and reflectivity maps. These results enable the development of high-efficient and low-noise LiDAR systems.
© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Active optical imaging underpins the estimation of various scene in widespread applications by measuring time-varying light intensities at each location. Single-photon detection has become a well-established technique in active imaging. Unlike conventional cameras, single-photon detection (SPD) observes individual photon incidents, thus enabling reporting weak optical signals. Illustration of conventional and single-photon detection schemes is shown in Fig. 1. By exploiting this technique [1], single-photon light detection and ranging (LiDAR) systems offer excellent sensitivity and ability to time-tag photon arrivals with picosecond temporal resolution [2], thus playing a pivotal role especially in demanding imaging under extreme conditions such as photon-starved imaging [3], ultra-long-range imaging [4], non-line-of-sight imaging [5], and imaging through scattering media [6].
Fig. 1. Conventional pixel vs. single-photon pixel vs. photon-number-resolving pixel. Conventional pixel acquires an intensity I proportional to the energy for the incident photons. Single-photon pixel is sensitive to an individual photon incident, and records its timestamps t. Photon-number-resolving pixel enables photon number counting for a single-photon incident and outputs its corresponding timestamp t and photon number k limited to maximum resolvable photon number M.
A typical single-photon LiDAR system accumulates binary photon data over multiple illumination cycles, thus forming a histogram of photon counts with respect to time-of-flight (TOF) for each pixel. Ideally, the photon-counting histogram is scaled to the incident waveform and can therefore be used to estimate the depth and reflectivity from the 3D data cube. However, in realistic scenarios, it is challenging to reconstruct accurate and robust depth and reflectivity maps due to the limited signal photon counts returned by the observed target and high background noise. Therefore, considerable efforts have been made to address this problem for high-quality reconstruction including photon-efficient reconstruction algorithms [7–10] and low-noise imaging systems [11,12].
Moreover, quantum LiDAR has become one of the research hotspots in recent years due to its advantages such as extreme sensitivity and resolution as well as noise-tolerant capability. However, although the basic theory related to quantum LiDAR with a non-classical light source tends to mature [13,14], it is technically difficult to implement: in general, the preparation of entangled states is difficult and environmentally sensitive, and susceptible to losses and noise leading to decoherence. Therefore, the use of optical quantum detection theory with a conventional classical state of light offers an accessible alternative, and still exhibits quantum advantages over single-photon detection. Representatively, Lior Cohent et al. [15] proposed a threshold quantum LiDAR based on photon-number-resolving detectors which produce a multi-level output signal with dependence on the number of photons for a single detected event as shown in Fig. 1. The threshold detection (TD) scheme filters out low photon numbers below the threshold to retain the relevant signal photons and suppress the randomly distributed background noise, with a substantially improved signal-to-noise ratio compared to SPD. Such detection methods [16–19] are similar to “coincidence detection” proposed for multiple SPADs [20–22] which output a signal only when two or more detectors respond simultaneously, thus reducing the false alarm rate. However, these techniques are only effective under high-signal low-noise regimes and thresholds setting wastes information of multi-level output [23]. While under low signal levels, they fail mainly because limited signal photon counts cannot be extracted from noise.
In this manuscript, we propose a threshold photon-number-resolving detection (TPNRD) scheme, with combined advantages of noise suppression by setting the upper and lower thresholds and signal enhancement by retaining photon number levels. TPNRD shows the significant performance advantages over other schemes under a wide range of environmental conditions as quantified by signal-to-noise-ratio (SNR), detection rate and false alarm rate criterions. Furthermore, based on the Poisson distribution of photon-number-resolving measurements, we derive a maximum likelihood estimation model for depth and reflectivity. The imaging experiment results on the synthetic and real-world datasets match well with the theoretical predictions obtained from the above criterions.
2. Related work
2.1 Photon-efficient imaging algorithms
Existing photon-efficient imaging algorithms can be divided into two categories, including statistical methods under an optimization framework [7,8] and deep learning methods [9,10]. Although these methods can improve the quality of reconstruction, they are computationally complex which might be not applicable for practical applications, particularly in imaging scenarios where both speed and accuracy are required. In contrast, we adopt an improved detection scheme to achieve better results during the measurement process over a broad range of signal and noise levels. In addition, the proposed scheme can be applied in combination with photon-efficient algorithms to alleviate the deteriorating effects of noise on algorithms.
2.2 Optical system architecture optimization
Generally, it is possible to distinguish signal from noise by long acquisition time and high laser frequency (i.e., increasing the number of detection cycles), which limit their use in high-speed and long-range imaging, respectively, and do not work at low SNR conditions. Therefore, one of the main objectives of optimizing the hardware architecture is to improve the SNR. Although a smaller receiving field of view and aperture can lead to substantially reduced ambient light with higher resolution, it is not conducive to detection of weak echo signals. To solve this problem, the laser intensity needs to be increased, signifying a high demand for light source. The strategy proposed is different, enabling high-SNR imaging of the same optical machine structure.
2.3 Photon-number-resolving detector
Photon-number-resolving detectors can provide more information [24] and is what allow it to be increasingly used in a variety of applications [25–28], such as quantum information, fluorescence imaging and deep-space communication. Photon-number-resolving detectors could be realized in three ways, including temporal or spatial multiplexing techniques [29–31], and individual detectors that generate pulses proportional to the number of absorbed photons, such as superconducting transition-edge sensors (TES) [32] or avalanche photodiode in the sub-saturated mode [33]. With the development of photon-number-resolving detectors, performance such as photon number resolution (i.e., the maximum resolvable photon number) and operation frequency has been enhanced [34,35], providing technical support for better results. Our demonstration is based on realistic photon-number-resolving detectors with finite photon number resolution, and is used to validate the impact of greater photon number resolution on the performance improvements under higher flux conditions shown by Fig. S1 in Supplement 1.
3. Threshold photon-number-resolving detection
3.1 Signal-to-noise ratio
During daylight, in addition to the pulse-echo laser, the photons received by LiDAR also arise from ambient background noise and detector dark counts. The strong radiation noise from the sun plays a dominant role, thus it is assumed that dark counts could be ignored to the later theoretical derivation. Moreover, we assume that the dead time of single-photon detectors and photon-number-resolving detectors is up to a few hundred nanoseconds or even microseconds, thus responding to at most one photon incident per period. In order to quantify the performance of different detection schemes, SNR is introduced as an evaluation metric which is defined as the distance between the mean signal $S + N$ and the mean background noise N divided by the standard deviation of noise [22,36], where noise refers to the fluctuation of both background noise and signal:
where $E({\cdot} )$ and $\sigma ({\cdot} )$ denote the mean value and the standard deviation, respectively.The background noise is thermal radiation source which obeys the Bose-Einstein distribution. However, for single-photon detectors and photon number-resolving detectors, each detection cycle is much longer than coherence time of thermal light. The rise and fall of the thermal light cannot be identified, and the measurement over multiple cycles results in an average of light intensity fluctuations [37], which can be considered as constant. Therefore, the background noise photon ${k_n}$ has a Poisson photon distribution:
where is the average noise photon number. The laser also obeys a Poisson distribution with the average signal photon number ${u_s}$. Since the detected signal is a mixture of coherent laser and ambient noise, signal photons k has mixed statistics, which can be simplified as (see Eq. (S1) in Supplement 1 for details):It still obeys a Poisson distribution with a mean of $u = {u_s} + {u_n}$, where u reflects the level of incident flux for each cycle.
SPD can only provide a bistatic output of “no photon (0)” or “photons (1)”. Therefore, its signal probability is ${p^{\prime}_s} = 1 - p({k = 0} )= 1 - {e^{ - u}}$ and noise probability is ${p^{\prime}_n} = 1 - {p_n}({k = 0} )= 1 - {e^{ - {u_n}}}$, both obeying Bernoulli distribution. The SNR of SPD over L cycles can be easily obtained from the mean and standard deviation of the n-fold Bernoulli distribution:
TD [15] also outputs a bistatic result of “photon number is less than threshold (0)” or “photon number is greater than ${p^{\prime\prime}_s} = p({k \ge {N_{\min }}} )= 1 - p({k\mathrm{ < }{N_{\min }}} )= 1 - {e^{ - u}}\sum\nolimits_{k = 0}^{{N_{\min }} - 1} {{u^k}/k!}$ threshold (1)”. Thus, its signal probability is and noise probability is ${p^{\prime\prime}_n} = 1 - {e^{ - {u_n}}}\sum\nolimits_{k = 0}^{{N_{\min }} - 1} {({{u_n}^k/k!} )}$, where ${N_{\min }}$ represents the lower threshold. ${N_{\min }}$ is assumed to be greater than 2, because TD is equivalent to SPD when ${N_{\min }} = 1$. Similar to Eq. (4), we get that the SNR of TD over L cycles is
Since TD surpasses SPD at the price of renouncing multiple levels of information, we propose the TPRND scheme. TPRND identifies photon number levels to realize the full capability of photon-number-resolving detectors and is further combined with the advantages of TD at high-signal low-noise levels. In brief, TPRND sets a lower threshold ${N_{\min }}$ and an upper threshold ${N_{\max }}$, and then preserves all photons number between two thresholds, as shown below:
From Eq. (10), it can be seen that the SNR suffers from the lower threshold ${N_{\min }}$, upper threshold ${N_{\max }}$, average signal photon number ${u_s}$, and average noise photon number ${u_n}$. In Supplement 1, Fig. S2 and Fig. S3, we evaluate effects of different lower and upper thresholds on SNR of TPRND.
We demonstrate that SNR could be much improved by TPRND as compared with other detection schemes in Fig. 2(a) and (b). Although there is little distinction when comparing SNR of TPRND and TD at high-signal low-noise levels, our scheme offers a clear benefit as it achieves up to 2 times higher SNR than that of TD in other cases. Compared with SPD, the proposed scheme achieves a substantial improvement over a wide range of ambient and illumination conditions except for the extreme low flux levels where photon number cannot be identified. Additionally, Fig. 2(c) shows the variation curves of SNR when the average noise or signal photon number is fixed at 1. TD applies to high-signal low-noise levels and TPRND without thresholding applies to high-noise low-signal levels, while TPRND combines benefits of both and is definitely prefer under all conditions.
Fig. 2. Comparison of SNR of different detection schemes: (a) the ratio of SNR with TPRND and with TD, (b) the ratio of SNR with TPRND and with SPD. All SNRs with thresholding are optimized with the most appropriate threshold. The average signal photon number ${u_s}$ and noise photon number ${u_n}$ vary from 0.1 to 10. The red and blue areas indicate ratios less than 1 and greater than 1, respectively. (c) (Top) The variation curves of SNR with increasing signal for ${u_n} = 1$, and (Bottom) the variation curves of SNR with increasing noise for ${u_s} = 1$. The green, red, and yellow areas represent the advantageous regions of the TD, TPRND without thresholding and SPD, respectively, and the purple region shows the further improvement of TPRND compared with other schemes.
3.2 Detection rate and false alarm rate
In Section 3.1, the Poisson photon distribution and SNR for different detection schemes are derived using the average signal and noise photon number. In photon-counting LiDAR, time-correlated single-photon counting (TCSPC) records photon incidents as time-of-flight, which can retain detection information adequately than the number of photons. Therefore, this section combines Poisson distribution with time-of-flight to derive a time-dependent detection model and to analyse the detection rate and false alarm rate of different schemes.
From Eq. (3), the statistics of the detected signal photons per cycle is given by a Poisson distribution, i.e., $k \sim Possion({\lambda (t )} )$, as $\lambda (t )$ represents the average photon rate
where $\lambda_s (t )$ denotes the time-varying photon rate of pulse-echo laser and ${\lambda _n}$ denotes the ambient photon rate independent of time, represented by $t \in \{{1,\ldots ,T} \}$, where $T = 1/f/\Delta $, $f$ and $\Delta $ are the laser frequency and the width of time bins, respectively. The signal photon rate ${\lambda _s}(t )$ is mainly concentrated within the narrow full width at half maximum (FWHM) of the laser pulse ${p_w}$. We assume that ${p_w} \leq {\Delta}$ and signal photons are detected only in the target bin ${T_P}$. Therefore, $\int_0^T {{\lambda _s}(t )dt} = \int_{t = {T_P}} {{\lambda _s}(t )dt} = {u_s}$, $\int_0^T {{\lambda _n}dt} = {\lambda _n}T = {u_n}$.For SPD and TPRND without thresholding, the probability that no photon is detected during the time ${t_1} \sim {t_2}$ is:
where $M({{t_1},{t_2}} )= \int_{{t_1}}^{{t_2}} {\lambda (t )} dt$ denotes the average photon number during the time . Due to the existence of dead time, the probability of detecting a photon during the time ${t_1} \sim {t_2}$ is predicated on not detecting a photon during the time $0 \sim {t_1}$. Therefore, the probability of detecting a photon during the time ${t_1} \sim {t_2}$ isFor TD and TPRND, the lower threshold affects the detection rate and false alarm rate. Therefore, the probability of detection during the time ${t_1} \sim {t_2}$ is
According to Eq. (10) and (11), we derive the detection probability models of different detection schemes in time bin t as shown in Table 1:
Table 1. Detection probability of different detection schemes in time bin $t$
The detection rate is defined as the probability of detecting a photon in the target bin ${T_P}$, i.e., ${P_D} = P({t = {T_P}} )$. For schemes without thresholding, the false alarm rate is defined as the probability of detecting a photon in the non-target bin ${T_P}$, i.e., ${P_F} = 1 - P({t = {T_P}} )- {P_{k = 0}}({1,T} )$, where ${P_{k = 0}}({1,T} )= {e^{ - {u_n} - {u_s}}}$. For schemes with thresholding, the false alarm rate is defined as ${P_{F\_T}} = 1 - {P_T}({t = {T_P}} )- {P_{k < {N_{\min }}}}({1,T} )$, where ${P_{k < {N_{\min }}}}({1,T} )= \sum\nolimits_{k = 0}^{{N_{\min }} - 1} {{{({{u_n} + {u_s}} )}^k}{e^{ - {u_n} - {u_s}}}} /k!$.
We firstly evaluate effects of the lower threshold ${N_{\min }}$ on detection rate and false alarm rate, as shown in Fig. 3(a). As flux level increases, taking the lower threshold larger enables better detection rate. Compared with its influence on SNR in Fig. 2(a), the lower threshold for optimal detection rate is smaller than that for optimal SNR at the same level. Thus, the lower threshold designed by Eq. (7) may optimize SNR at the price of deteriorating detection rate. Larger threshold produces better false alarm rate, and the optimal threshold is proportional to the average photon number of noise only at extremely high signal levels. Figure 3(b) shows the difference of the detection rate and false alarm rate of schemes with thresholding and without thresholding. The improvement in detection rate and false alarm rate become greater with increasing incident flux and with increasing noise levels, respectively.
Fig. 3. (a) Effects of the lower threshold on the detection rate and false alarm rate. (b) Difference of the detection rate and false alarm rate of schemes with thresholding and without thresholding. The average signal photon number ${u_s}$ and noise photon number ${u_n}$ vary from 0.1 to 10. Optimal rates ${P_{D\_T}}$ and ${P_{F\_T}}$ are obtained with appropriate lower thresholds. Regions where the difference is less than 0 and greater than 0 are marked in red and blue, respectively. It is assumed that $T = 512bins$ and the target locates in the middle of a cycle, i.e., ${T_P} = T/2$. Schemes with thresholding improve the detection rate and false alarm rate of schemes without thresholding.
In conclusion, TPRND proposed in this manuscript can enhance SNR and the detection rate, and lower the false alarm rate. However, since the lower thresholds for optimal SNR, detection rate and false alarm rate differ, we comprehensively consider influences of these three criterions and design the lower threshold and upper threshold to achieve acceptable detection results, as follows:
4. Imaging model
In this section, to visualize and compare the imaging results of different detection schemes, we formulate the imaging model of TPRND-based LiDAR for the first time. This section first introduces the data acquisition process for systems based on different detection schemes. Then, the observation model and likelihood approximation of TPRND-based LiDAR are derived to reconstruct the depth and reflectivity maps.
4.1 Data acquisition process
A typical LiDAR system accumulates photon counts over multiple laser pulse cycles, thus forming a histogram of photon counts with respect to time-of-flight (TOF) for each pixel, as shown in Fig. 4. Single-photon detectors, such as photomultiplier tubes (PMT) or single-photon avalanche photodiodes (SPAD), can only provide a bistatic output of “no photon (0)” or “photons (1)”. That is, such a SPD-based LiDAR records at most one photon event per cycle, and its timestamp is collected to add the counts in the corresponding histogram time bin. Ideally, the photon-counting histogram ${\boldsymbol h = }\{{{h_t}|{1 \le t \le T} } \}$ is scaled to the incident waveform and can therefore be used to estimate the depth and reflectivity, where ${h_t}$ represents photon counts of time bin t. In contrast, LiDAR based on TD or TPRND detection scheme exhibits the photon-number-resolving detectors which output pulses proportional to the energy of incident photons. The discrimination level of TCSPC acts on the output pulses of detectors, which keeps very low for SPD and correspond to the photon number of lower threshold for TD and TPRND. Since the TD-based LiDAR still outputs a binary result per cycle, the resulting histogram is formally consistent with and ideally a denoised version of the histogram obtained by SPD-based system. TPRND-based system yields a photon detection result in each cycle containing two features with time-of-flight and the photon number corresponding to the output pulse, and eventually establishes a 3D histogram for a pixel. The 3D histogram can be downscaled by accumulating counts according to the corresponding time bin, illustrated in the next section, and is ideally equivalent to noise suppression and signal enhancement for histograms of SPD-based systems. Eventually, histogram for a pixel is combined with 2-D scanning to give the observation data of the whole target scene.
Fig. 4. (a) Imaging schematic and (b-f) data acquisition process of the LiDAR system based on different detection schemes: (b) The laser emits periodic light pulses that are deflected by a two-dimensional fast steering mirror (FSM) and reflected to the detector. (c) The detector outputs an electrical pulse upon a photon arriving and enters a dead time ($\tau$) per cycle, during which additional photon cannot be detected. (d) Photon events or numbers are recorded after setting the discrimination level of TCSPC and the photon time-of-arrival histogram for one pixel is constructed after L cycles, where the time axis of the histogram is discretized into time bins ($\Delta $) whose length is determined by the temporal resolution of the TCSPC. (e) Eventually, the histograms of all pixels form a three-dimensional (3D) data cube, thus reconstructing the entire target scene. (f) The signal and noise are indicated in blue and orange. ${V_{OH}}$ and ${V_{OL}}$ indicate the high and low logic levels of the electrical output pulse.
4.2 Observation model and likelihood approximation
As described in Section 4.1, L cycles of TPRND-based LiDAR will yield L photon detection results, each sample of which is characterized by ${s_{n,l,t}}$, representing the photon counts at pixel position $n \in \{{1,\ldots ,N} \}$, cycle $l \in \{{1,\ldots ,L} \}$, and time bin $t \in \{{1,\ldots ,T} \}$ follows a Poisson distribution $P({\cdot} )$, which can be described as:
where: and ${\lambda _{n,t}}$ stands for the average photon counts of the $t$-th time bin at the pixel n, which can be considered as independent of cycle, $g(t) = \alpha {e ^{ - {{({t - {t_n}} )}^2}/({2{\sigma^2}} )}}$ represents the Gaussian approximation of the system impulse response of the LiDAR, where $\alpha$ and $\sigma$ are constant parameters ${r_n}$ and ${t_n}$ denote the reflectivity and photon time-of-flight corresponding to the depth of the target ${z_n}$, respectively, and ${b_n}$ represents the constant counts arising from background noise.Assuming the independency of each pixel, cycle and time bin, the joint probability distribution can be obtained by combining Eq. (13) and Eq. (14):
From Eq. (17), it can be seen that the estimates of depth and reflectivity are independent of each other. As a result, the maximum likelihood estimate (MLE) of photon time-of-flight at the pixel n is given by:
Similarly, the maximum likelihood estimate of reflectivity is given by:
The maximum likelihood estimators given by Eq. (18) and Eq. (19) analyze L histograms to reconstruct the depth map ${{\boldsymbol d}^{ML}}{\boldsymbol = c}{{\boldsymbol t}^{ML}}/2$ and reflectivity map ${{\boldsymbol r}^{ML}}$, where c denotes the velocity of light. Additionally, these estimators can be deformed to acquire $t_n^{ML} = \sum\nolimits_{t = 1}^T {t{{\tilde{s}^{\prime}}_{n,t}}} /\sum\nolimits_{t = 1}^T {{{\tilde{s}^{\prime}}_{n,t}}}$ and $r_n^{ML} = \sum\nolimits_{t = 1}^T {{{\tilde{s}^{\prime}}_{n,t}}} /L$, where ${\tilde{s}^{\prime}_{n,t}} = \sum\nolimits_{l = 1}^L {{{\tilde{s}}_{n,l,t}}}$, equivalent to accumulating the detection results of L cycles on each pixel together according to the corresponding time bin and compressing them into a 2-D histogram ${\tilde{s}^{\prime}_{n,t}}$. These deformations inspire us: (1) The compressed histogram can preserve the photon number information, and its data dimension consistent with the histogram of SPD schemes makes it compatible with state-of-the-art photon-efficient algorithms, further improving estimating performance. (2) To reduce the size of observation data, detection results of the photon number of each collected pulse over L cycles can be stored in a two-dimensional histogram by accumulating counts according to the corresponding time bin.
5. Evaluation on synthetic data
This section successively introduces the design of synthetic data, evaluation metrics, and analysis of imaging results on the synthetic data compared with other sachems.
5.1 Design of synthetic data
Referring to the data acquisition process in Fig. 4 and the synthetic data simulation method in [10], we design synthetic data from different detection schemes: Firstly, the incident light rate function as illustrated in Eq. (14) is obtained from the Middlebury dataset [40], and the parameters signal rate and noise rate are scaled to achieve a given signal and noise level. Secondly, random numbers obeying an inhomogeneous Poisson distribution are generated, as illustrated in Eq. (13), and the first non-zero random number is chosen with probability $\eta$ to acquire the photon number and time-of-flight per cycle, where probability $\eta$ denotes the system efficiency assumed as 1. Thirdly, the number of photons is compared with the lower and upper thresholds obtained by Eq. (12), that is ${s_{n,l,t}} = \min ({k,{N_{\max }}} )\textrm{ }if\textrm{ }k \ge {N_{\min }}$, where ${N_{\max }} \le M\textrm{ = }4$.
Finally, the full data of the proposed TPRND denoted by ${\boldsymbol s = }\{{{s_{n,l,t}}|{1 \le n \le N,1 \le l \le L,1 \le t \le T} } \}$ is collected after L cycles. Furthermore, to compare the performance of different detection schemes under the same conditions, we consider the data of SPD, TD and TPRND obtained from the same preserved random number.
5.2 Evaluation metrics
Comparison is based on the metrics including root mean square error (RMSE) and structural similarity (SSIM) for quantitative evaluation of depth and reflectivity maps:
5.3 Results
The synthetic data of “artistic” scene, is generated by utilizing an image size of 144 × 176 pixels, 521 time bins corresponding to 100 ns, an illumination pulse with a FWHM of 270 ps, a maximum resolvable photon number $M = 4$ and various photons per pixel (PPP) levels by setting different cycles L. The PPP of SPD is approximately equal to the number of cycles L. Thresholds for TD and TPRND are obtained by Eq. (12) and $4 \ge {N_{\max }} \ge {N_{\min }} \ge 1$.
Figure 5 plots histograms of different detection schemes at two representative levels (${u_s}:{u_n} = 5:2,2:10$) and number of cycles ($L = 10,2$). We present results of TPRND without thresholding separately, i.e., TPRND with ${N_{\max }} = 4$ and ${N_{\min }} = 1$, to show the impact of thresholds. At the high-signal low-noise level (${u_s}:{u_n} = 5:2$), TPRND with or without thresholding can substantially enhance the signal compared to SPD and TD. While at the low-signal high-noise level (${u_s}:{u_n} = 2:10$), thresholding effectively suppresses noise due to ambient light but result in heavy signal loss for TD. In contrast, TPRND can retain more signal, and the ratio of the signal peak to the noise peak arriving before the laser pulse is further increased from 35.17 to 50.90 as the number of cycles increases from 2 to 10, indicating that it can further enhance signal without incurring more noise. In addition, the ratio of SPD is much smaller than that of TPRND and differs slightly as the number of cycles increases. This phenomenon suggests that the histograms obtained by SPD suffer from severe nonlinear distortions, i.e., “pile-up,” and cannot be mitigated by increasing the number of cycles but can by TPRND.
Fig. 5. Histograms of the pulse area of length 100 for all pixels acquired with different detection schemes at different levels (${u_s}:{u_n} = 5:2,2:10$) and number of cycles ($L = 2,10$). TPRND can retain more signal and suppress noise, resulting in the ratio of the signal peak to the noise peak arriving before the laser pulse much larger than that of other schemes and further improved as the number of cycles increases.
Depth reconstruction results of different schemes under a variety of signal and noise levels and detection cycles are shown in Fig. 6. The first row shows MLE results under the condition of high signal and low noise, since in this case MLE can achieve good results. The rest of the rows utilize a neural-network-based algorithm, adopted from our previous work [41], which adds attention-guided fusion modules with gradient regularization functions to a multi-scale integrated U-Net network to achieve highly accurate depth reconstruction. When using the same estimation method under the same conditions, i.e., MLE method for ${u_s}:{u_n} = 5:2$ and the neural-network-based algorithm for ${u_s}:{u_n} = 2:10$, more detection cycles lead to better imaging results. TPRND successfully achieves the optimal RMSE and SSIM, even enabling recovery of fine details at all levels. At the high-signal low-noise level (${u_s}:{u_n} = 5:2$), the visual and numerical results obtained by TPRND with 10 cycles are better than those obtained by SPD with 50 cycles, making potential use of the proposed scheme in high-speed imaging application. At the low-signal level (${u_s}:{u_n} = 1:5$), TD loses most of signal information which although can be recovered by [41], resulting in blurred edges. At high-noise levels (${u_s}:{u_n} = 2:10,2:20$), TPRND and TD with thresholding suppress both signal and noise, leading to inevitable signal loss, but problem is exacerbated for SPD that suffers from strong effects of pile-up and most of the scene depth is shifted forward although only photon counts within the target depth range are preserved. MLE results of reflectivity obtained with different schemes are shown in Fig. 7. TPRND preforms better than other schemes at different levels, and reconstructs smoother reflectivity maps that get closer to the ground truth with clearer boundary and finer details with cycles increasing. We also visualize effects of different lower and upper thresholds on histogram and depth reconstruction of TPRND at different signal levels in Supplement 1, Fig. S4.
Fig. 6. Depth reconstruction results of different schemes under a variety of signal and noise levels (${u_s}:{u_n} = 5:2,2:2,2:10,1:5,2:20$) and detection cycles ($L = 2,10,50$). The first row and the rest of the rows show the results of MLE and photon-efficient algorithm based on a neural network, respectively. TPRND successfully achieves the optimal visual and numerical results at most levels.
Fig. 7. Reflectivity reconstruction results of different schemes under a variety of signal and noise levels (${u_s}:{u_n} = 5:2,2:2,2:10,1:5,2:20$) and detection cycles ($L = 50,200$). All estimates are obtained from MLE of the data within the target depth range. Thresholds for TD and TPRND are obtained by Eq. (12). TPRND preforms better than other schemes at different levels qualitatively and quantitatively.
6. Evaluation on real-world data
6.1 Experimental equipment
The schematic diagram of the TPRND-based photon counting LiDAR system is similar to that shown in Fig. 4. The light source emits laser pulses with a frequency of 10 kHz and a wavelength of 1550 nm, which is deflected by a two-axis fast steering mirror to scan the entire target scene. The four-channel superconducting nanowire single photon detectors (SNSPDs) with maximum resolvable photon number $M = \textrm{4}$ receives laser echoes from the target and background light. The TCSPC module (PicoQuant HydraHarp 400) records photon incidents and deduces the time-of-flight for each photon. Detailed hardware implementation and key parameters are shown in Section S3 in Supplement 1.
6.2 Experimental results
We capture the data of scene at 1.2 km in the high-signal low-noise case as shown in Fig. 8(a) and compare results of all detection schemes. The upper threshold ${N_{\max }}$ is set to 4 and we present results of TPRND with ${N_{\min }} = 1$ and ${N_{\min }} = 2$, to show the impact of different lower thresholds in real-world applications. The depth reconstruction algorithm used here is a pixel-by-pixel time-domain windowing module of our previous work [39], to suppress ambient noise of outdoor scenes imaging according to the signal-to-background ratio (SBR), where the size of window is set to 3 times FWHM and SBR is defined as the ratio of the number of signal photons to the number of noise photons per pixel. At low noise levels, TPRND achieve very sharp prominent peaks in photon-counting histograms by enhancing the signal compared to TD and SPD as shown in Fig. 8(b).
Fig. 8. Target at 1.2 km in the high-signal low-noise case: (a) visible-band, image and (b) photon-counting histogram. The FWHM of the illumination pulses is 7 ns and the number of time bins is 1000, corresponding to 400 ns. SBRs of the remaining detection schemes are $\textrm{SB}{\textrm{R}_{\textrm{TD}}}\textrm{ = 1}\textrm{.4}:\textrm{0}\textrm{.2}$ and $\textrm{SB}{\textrm{R}_{\textrm{TPRND}({{N_{\min }} = 2} )}}\textrm{ = 4}\textrm{.9}:\textrm{0}\textrm{.4}$. The proposed TPRND obtain more prominent signal peaks than other schemes, and TPRND with ${N_{\min }} = 2$ achieves the optimal SBR with 12.25 times and 1.75 times improvement over SPD and TD, respectively.
Figure 9 compares the qualitative and quantitative results. The proposed TPRND with ${N_{\min }} = 2$ and ${N_{\min }} = 1$ show optimal visual and numerical performance when using maximum likelihood estimator and the reconstruction algorithm respectively. When only applying MLE, TPRND with ${N_{\min }} = 2$ performs best and TD likewise gives comparable results. In this case, SPD and TPRND with ${N_{\min }} = 1$ cannot filter out noise, but TPRND with ${N_{\min }} = 1$ can still exhibit less noisy pixels than SPD. With the reconstruction algorithm, TPRND with ${N_{\min }} = 1$ optimally restore fine details of the scene, followed by TPRND with ${N_{\min }} = 2$. In contrast, SPD and TD cannot enhance the signal, resulting in slight difference between signal and noise, thus filtering out the noise will inevitably cause some structures to be missing. Furthermore, depth maps obtained with TD and TPRND with ${N_{\min }} = 2$ using the reconstruction algorithm become less noisy, but with more missing pixels and deteriorating numerical results compared to MLE results, since the larger lower threshold loses some signal information before the algorithm. From the reflectivity reconstruction results we find that TPRND can better distinguish between different targets, such as walls and windows, highlighted in the red frame. Moreover, when comparing results in Fig. 8 and Fig. 9, the advantage of TPRND in pixel-level reconstruction performance is not as evident as in the combined histogram results of all pixels. This is because when the detection cycles are small and the echo pulses are weak (i.e. small SBR) or maximum resolvable photon number is small, the signal amplification capability of TPRND is limited at individual pixels. But the improvement can be enhanced with the increase of detection cycle and the number of resolvable photons. Considering the distinct advantage displayed by TPRND in the histogram, it also facilitates localizing the range of the scene and can be further enhanced when combined with algorithms based on spatial correlation. In addition to high SBR scenes, we also capture real-world data in the low-signal high-noise case as illustrated in Fig. S6 and Fig. S7 in Supplement 1.
Fig. 9. Reconstruction results in the high-signal low-noise case: depth reconstruction results (Top) with MLE and (Middle) with the reconstruction algorithm, and (Bottom) reflectivity reconstruction results with the maximum likelihood estimator. The image size is 200 × 200. GT indicates the ground-truth map captured with longest acquisition time. The proposed TPRND show optimal performance.
7. Conclusion
In this manuscript, we present TPRND, which is a quantum-enhanced LiDAR detection scheme by retaining all photon number levels between the upper and lower threshold. Benefiting from the combined capability of signal enhancement and noise suppression, TPNRD demonstrates superior performance compared to SPD and TD, as quantified by SNR, detection rate, and false alarm rate. Experiments on synthetic datasets verify the excellent depth and reflectivity reconstruction results visually and numerically, even with extreme few detection cycles or under high-noise low-signal conditions. In addition, the proposed scheme is tested in real-world experiments. To conclude, TPRND enhance the LiDAR performance over a broad range of signal and noise levels. With the shortening of the dead time of photon-number-resolving detectors and TCSPC, multiple photon incidents could be recorded in one cycle, indicating a potential direction for discussing the performance of such TPRND-based LiDAR systems. Furthermore, future research should be devoted to proposing a unified and reliable quantitative evaluation method to comparing various detection technologies based on multiple detectors [15–22,38,42] in the context of LiDAR to further verify the advantages of TPRND.
Funding
National Natural Science Foundation of China (61875088, 62005128).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
Supplemental document
See Supplement 1 for supporting content.
References
1. R. H. Hadfield, “Single-photon detectors for optical quantum information applications,” Nat. Photonics 3(12), 696–705 (2009). [CrossRef]
2. B. Vojnovic, “Advanced time-correlated single photon counting techniques,” J. Microsc. 222(1), 65–66 (2006). [CrossRef]
3. A. Kirmani, D. Venkatraman, D. Shin, et al., “First-Photon Imaging,” Science 343(6166), 58–61 (2014). [CrossRef]
4. Z. P. Li, J. T. Ye, X. Huang, et al., “Single-photon imaging over 200 km,” Optica 8(3), 344–349 (2021). [CrossRef]
5. C. Wu, J. Liu, X. Huang, et al., “Non–line-of-sight imaging over 1.43 km,” Proc. Natl. Acad. Sci. U. S. A. 118(10), e2024468118 (2021). [CrossRef]
6. R. Tobin, A. Halimi, A. McCarthy, et al., “Three-dimensional single-photon imaging through obscurants,” Opt. Express 27(4), 4590–4611 (2019). [CrossRef]
7. J. Rapp and V. K. Goyal, “A few photons among many: Unmixing signal and noise for photon-efficient active imaging,” IEEE Trans. Comput. Imaging 3(3), 445–459 (2017). [CrossRef]
8. A. Halimi, A. Maccarone, R. A. Lamb, et al., “Robust and guided bayesian reconstruction of single-photon 3d lidar data: Application to multispectral and underwater imaging,” IEEE Trans. Comput. Imaging 7, 961–974 (2021). [CrossRef]
9. Z. Sun, D. B. Lindell, O. Solgaard, et al., “SPADnet: deep RGB-SPAD sensor fusion assisted by monocular depth estimation,” Opt. Express 28(10), 14948–14962 (2020). [CrossRef]
10. D. B. Lindell, M. O’Toole, and G. Wetzstein, “Single-photon 3D imaging with deep sensor fusion,” ACM Trans. Graph. 37(4), 1–12 (2018). [CrossRef]
11. A. M. Pawlikowska, A. Halimi, R. A. Lamb, et al., “Single-photon three-dimensional imaging at up to 10 kilometers range,” Opt. Express 25(10), 11919–11931 (2017). [CrossRef]
12. Z.-P. Li, X. Huang, Y. Cao, et al., “Single-photon computational 3D imaging at 45 km,” Photonics Res. 8(9), 1532–1540 (2020). [CrossRef]
13. J. Zhao, A. Lyons, A. C. Ulku, et al., “Light detection and ranging with entangled photons,” Opt. Express 30(3), 3675–3683 (2022). [CrossRef]
14. H. Liu, D. Giovannini, H. He, et al., “Enhancing LIDAR performance metrics using continuous-wave photon-pair sources,” Optica 6(10), 1349–1355 (2019). [CrossRef]
15. L. Cohen, E. S. Matekole, Y. Sher, et al., “Thresholded quantum LIDAR: exploiting photon-number-resolving detection,” Phys. Rev. Lett. 123(20), 203601 (2019). [CrossRef]
16. B. Zhang, Y.-Q. Guan, L. Xia, et al., “An all-day lidar for detecting soft targets over 100 km based on superconducting nanowire single-photon detectors,” Supercond. Sci. Technol. 34(3), 034005 (2021). [CrossRef]
17. L. Cohen, D. Istrati, Y. Sher, et al., “Laser Ranging Bathymetry Using a Photon-Number-Resolving Detector,” Remote Sens. 14(19), 4750 (2022). [CrossRef]
18. Z. Bao, Y. Liang, Z. Wang, et al., “Laser ranging at few-photon level by photon-number-resolving detection,” Appl. Opt. 53(18), 3908–3912 (2014). [CrossRef]
19. R. Yang, Y. Tang, Z. Fu, et al., “A Method of Range Walk Error Correction in SiPM LiDAR with Photon Threshold Detection,” Photonics 9(1), 24 (2022). [CrossRef]
20. L. Ye, G. Gu, W. He, et al., “A reconstruction method for restraining range walk error in photon counting Lidar via dual detection,” J. Opt. 21(4), 045703 (2019). [CrossRef]
21. Z. Zhang, Y. Zhao, Y. Zhang, et al., “A real-time noise filtering strategy for photon counting 3D imaging lidar,” Opt. Express 21(8), 9247–9254 (2013). [CrossRef]
22. A. Tontini, L. Gasparini, E. Manuzzato, et al., “Comparative evaluation of background-rejection techniques for SPAD-based LiDAR systems,” Integration 90, 1–10 (2023). [CrossRef]
23. E. Lantz, “Comment on “Thresholded Quantum LIDAR: Exploiting Photon-Number-Resolving Detection”,” Phys. Rev. Lett. 124(19), 199302 (2020). [CrossRef]
24. F. Becerra, J. Fan, and A. Migdall, “Photon number resolution enables quantum receiver for realistic coherent optical communications,” Nat. Photonics 9(1), 48–53 (2015). [CrossRef]
25. M. Cattaneo, M. G. Paris, and S. Olivares, “Hybrid quantum key distribution using coherent states and photon-number-resolving detectors,” Phys. Rev. A 98(1), 012333 (2018). [CrossRef]
26. E. Slenders, M. Castello, M. Buttafava, et al., “Confocal-based fluorescence fluctuation spectroscopy with a SPAD array detector,” Light: Sci. Appl. 10(1), 31 (2021). [CrossRef]
27. X. Li, J. Tan, K. Zheng, et al., “Enhanced photon communication through Bayesian estimation with an SNSPD array,” Photonics Res. 8(5), 637–641 (2020). [CrossRef]
28. S. Li, W. Li, V. V. Yakovlev, et al., “En route to nanoscopic quantum optical imaging: counting emitters with photon-number-resolving detectors,” Opt. Express 30(8), 12495–12509 (2022). [CrossRef]
29. K. Banaszek and I. A. Walmsley, “Photon counting with a loop detector,” Opt. Lett. 28(1), 52–54 (2003). [CrossRef]
30. D. Achilles, C. Silberhorn, C. Śliwa, et al., “Fiber-assisted detection with photon number resolution,” Opt. Lett. 28(23), 2387–2389 (2003). [CrossRef]
31. A. Divochiy, F. Marsili, D. Bitauld, et al., “Superconducting nanowire photon-number-resolving detector at telecommunication wavelengths,” Nat. Photonics 2(5), 302–306 (2008). [CrossRef]
32. Y. Mitsuya, T. Konno, S. Takasu, et al., “Photon number resolution with an iridium optical transition edge sensor at a telecommunication wavelength,” J. Low Temp. Phys. 210(3-4), 498–505 (2023). [CrossRef]
33. G. Wu, Y. Jian, E. Wu, et al., “Photon-number-resolving detection based on InGaAs/InP avalanche photodiode in the sub-saturated mode,” Opt. Express 17(21), 18782–18787 (2009). [CrossRef]
34. J. Lin, Y. Sun, W. Wu, et al., “High-speed photon-number-resolving detection via a GHz-gated SiPM,” Opt. Express 30(5), 7501–7510 (2022). [CrossRef]
35. M. Schmidt, M. Von Helversen, M. López, et al., “Photon-number-resolving transition-edge sensors for the metrology of quantum light sources,” J. Low Temp. Phys. 193(5-6), 1243–1250 (2018). [CrossRef]
36. A. T. Pedersen, C. F. Abari, J. Mann, et al., “Theoretical and experimental signal-to-noise ratio assessment in new direction sensing continuous-wave Doppler lidar,” in Journal of Physics: Conference Series (IOP Publishing, 2014), p. 012004.
37. S. Kou, “Study of APD photon counting imaging technology,” Ph.D. dissertation (Nanjing University of Technology, 2010).
38. S. M. Patanwala, I. Gyongy, H. Mai, et al., “A high-throughput photon processing technique for range extension of SPAD-based LiDAR receivers,” IEEE Open J. Solid-State Circuits Soc. 2, 12–25 (2022). [CrossRef]
39. M. Wu, Y. Lu, H. Li, et al., “Intensity-guided depth image estimation in long-range lidar,” Opt. Lasers Eng. 155, 107054 (2022). [CrossRef]
40. D. Scharstein and C. Pal, “Learning conditional random fields for stereo,” in 2007 IEEE conference on computer vision and pattern recognition (IEEE, 2007), pp. 1–8.
41. X. Zhao, X. Jiang, A. Han, et al., “Photon-efficient 3D reconstruction employing a edge enhancement method,” Opt. Express 30(2), 1555–1569 (2022). [CrossRef]
42. P. Padmanabhan, C. Zhang, M. Cazzaniga, et al., “7.4 A 256×128 3D-stacked (45 nm) SPAD FLASH LiDAR with 7-level coincidence detection and progressive gating for 100 m range and 10klux background light,” in 2021 IEEE International Solid-State Circuits Conference (ISSCC), vol. 64 (IEEE, 2021), pp. 111–113.
